Mercurial > p > roundup > code
annotate roundup/backends/back_anydbm.py @ 6823:fe0091279f50
Refactor session db logging and key generation for sessions/otks
While I was working on the redis sessiondb stuff, I noticed that
log_wanrning, get_logger ... was duplicated. Also there was code to
generate a unique key for otks that was duplicated.
Changes:
creating new sessions_common.py and SessionsCommon class to provide
methods:
log_warning, log_info, log_debug, get_logger, getUniqueKey
getUniqueKey method is closer to the method used to make
session keys in client.py.
sessions_common.py now report when random_.py chooses a weak
random number generator. Removed same from rest.py.
get_logger reconciles all logging under
roundup.hyperdb.backends.<name of BasicDatabase class>
some backends used to log to root logger.
have BasicDatabase in other sessions_*.py modules inherit from
SessionCommon.
change logging to use log_* methods.
In addition:
remove unused imports reported by flake8 and other formatting
changes
modify actions.py, rest.py, templating.py to use getUniqueKey
method.
add tests for new methods
test_redis_session.py
swap out ModuleNotFoundError for ImportError to prevent crash in
python2 when redis is not present.
allow injection of username:password or just password into redis
connection URL. set pytest_redis_pw envirnment variable to password
or user:password when running test.
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Sun, 07 Aug 2022 01:51:11 -0400 |
| parents | feb175271f3e |
| children | 60b37f632601 |
| rev | line source |
|---|---|
|
213
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
1 # |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
2 # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
3 # This module is free software, and you may redistribute it and/or modify |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
4 # under the same terms as Python, so long as this copyright message and |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
5 # disclaimer are retained in their original form. |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
6 # |
| 214 | 7 # IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR |
|
213
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
8 # DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
9 # OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
10 # POSSIBILITY OF SUCH DAMAGE. |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
11 # |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
12 # BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, |
|
d45384bc6420
Added the copyright/license notice to (nearly) all files...
Richard Jones <richard@users.sourceforge.net>
parents:
149
diff
changeset
|
16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
17 # |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
18 """This module defines a backend that saves the hyperdatabase in a |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
19 database chosen by anydbm. It is guaranteed to always be available in python |
|
440
de5bf4191f11
Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents:
431
diff
changeset
|
20 versions >2.1.1 (the dumbdbm fallback in 2.1.1 and earlier has several |
|
de5bf4191f11
Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents:
431
diff
changeset
|
21 serious bugs, and is not available) |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
22 """ |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
23 __docformat__ = 'restructuredtext' |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
24 |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
25 import os, marshal, re, weakref, string, copy, time, shutil, logging |
|
1809
bd127cafe3a8
Simplify backend importing, by moving the imports into the backend modules.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
1800
diff
changeset
|
26 |
|
5011
d5da643b3d25
Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
27 from roundup.anypy.dbm_ import anydbm, whichdb |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
28 from roundup.anypy.strings import b2s, bs2b, repr_export, eval_import, is_us |
|
2908
95813789cf70
translate UI message
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2906
diff
changeset
|
29 |
|
3589
1be293265e61
woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
30 from roundup import hyperdb, date, password, roundupdb, security, support |
|
6401
8bc5faeb7677
Make rev multilink expressions work for anydbm
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6399
diff
changeset
|
31 from roundup.mlink_expr import Expression |
|
2908
95813789cf70
translate UI message
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2906
diff
changeset
|
32 from roundup.backends import locking |
|
95813789cf70
translate UI message
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2906
diff
changeset
|
33 from roundup.i18n import _ |
|
95813789cf70
translate UI message
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2906
diff
changeset
|
34 |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
35 from roundup.backends.blobfiles import FileStorage |
|
6814
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
36 from roundup.backends import sessions_dbm |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
37 |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
38 try: |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
39 from roundup.backends import sessions_redis |
|
6820
feb175271f3e
redis import failure on python2 causes crash
John Rouillard <rouilj@ieee.org>
parents:
6814
diff
changeset
|
40 except ImportError: |
|
6814
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
41 sessions_redis = None |
|
3295
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3239
diff
changeset
|
42 |
|
5096
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
43 from roundup.backends.indexer_common import get_indexer |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
44 |
|
5725
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
45 from hashlib import md5 |
|
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
46 |
|
2633
a9e1fff1e793
I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
47 def db_exists(config): |
|
a9e1fff1e793
I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
48 # check for the user db |
|
2637
11811b313459
The demo.py script works again using the new configuration system.
Richard Jones <richard@users.sourceforge.net>
parents:
2633
diff
changeset
|
49 for db in 'nodes.user nodes.user.db'.split(): |
|
2736
402d6d556558
postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents:
2699
diff
changeset
|
50 if os.path.exists(os.path.join(config.DATABASE, db)): |
|
2633
a9e1fff1e793
I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
51 return 1 |
|
a9e1fff1e793
I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
52 return 0 |
|
a9e1fff1e793
I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
53 |
|
a9e1fff1e793
I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
54 def db_nuke(config): |
|
2736
402d6d556558
postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents:
2699
diff
changeset
|
55 shutil.rmtree(config.DATABASE) |
|
2633
a9e1fff1e793
I thought I committed this last night. Ho hum.
Richard Jones <richard@users.sourceforge.net>
parents:
2617
diff
changeset
|
56 |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
57 # |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
58 # Now the database |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
59 # |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
60 class Database(FileStorage, hyperdb.Database, roundupdb.Database): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
61 """A database for storing records containing flexible data types. |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
62 |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
63 Transaction stuff TODO: |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
64 |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
65 - check the timestamp of the class file and nuke the cache if it's |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
66 modified. Do some sort of conflict checking on the dirty stuff. |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
67 - perhaps detect write collisions (related to above)? |
|
5096
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
68 |
|
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
69 attributes: |
|
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
70 dbtype: |
|
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
71 holds the value for the type of db. It is used by indexer to |
|
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
72 identify the database type so it can import the correct indexer |
|
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
73 module when using native text search mode. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
74 """ |
|
5096
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
75 |
|
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
76 dbtype = "anydbm" |
|
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
77 |
|
5100
5775959d81a3
issue1926124: fix crash in roundup_admin migrate option wih anydbm
John Rouillard <rouilj@ieee.org>
parents:
5096
diff
changeset
|
78 # used by migrate roundup_admin command. Is a no-op for anydbm. |
|
5775959d81a3
issue1926124: fix crash in roundup_admin migrate option wih anydbm
John Rouillard <rouilj@ieee.org>
parents:
5096
diff
changeset
|
79 # but needed to stop traceback in admin. |
|
5775959d81a3
issue1926124: fix crash in roundup_admin migrate option wih anydbm
John Rouillard <rouilj@ieee.org>
parents:
5096
diff
changeset
|
80 db_version_updated = False |
|
5096
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
81 |
|
524
dce4c75bef5a
changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents:
475
diff
changeset
|
82 def __init__(self, config, journaltag=None): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
83 """Open a hyperdatabase given a specifier to some storage. |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
84 |
|
524
dce4c75bef5a
changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents:
475
diff
changeset
|
85 The 'storagelocator' is obtained from config.DATABASE. |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
86 The meaning of 'storagelocator' depends on the particular |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
87 implementation of the hyperdatabase. It could be a file name, |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
88 a directory path, a socket descriptor for a connection to a |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
89 database over the network, etc. |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
90 |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
91 The 'journaltag' is a token that will be attached to the journal |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
92 entries for any edits done on the database. If 'journaltag' is |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
93 None, the database is opened in read-only mode: the Class.create(), |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
94 Class.set(), Class.retire(), and Class.restore() methods are |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
95 disabled. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
96 """ |
|
3897
e2c2d91932ad
respect umask on filestorage backend (fixes [SF#744328])
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3872
diff
changeset
|
97 FileStorage.__init__(self, config.UMASK) |
|
6658
408fd477761f
Add i18n object to roundupdb.Database
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6427
diff
changeset
|
98 roundupdb.Database.__init__(self) |
|
524
dce4c75bef5a
changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents:
475
diff
changeset
|
99 self.config, self.journaltag = config, journaltag |
|
dce4c75bef5a
changed all config accesses...
Richard Jones <richard@users.sourceforge.net>
parents:
475
diff
changeset
|
100 self.dir = config.DATABASE |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
101 self.classes = {} |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
102 self.cache = {} # cache of nodes loaded or created |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
103 self.stats = {'cache_hits': 0, 'cache_misses': 0, 'get_items': 0, |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
104 'filtering': 0} |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
105 self.dirtynodes = {} # keep track of the dirty nodes by class |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
106 self.newnodes = {} # keep track of the new nodes by class |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
107 self.destroyednodes = {}# keep track of the destroyed nodes by class |
|
252
76c6994aa4e8
CGI interfaces now spit up a top-level index of all instances they can serve.
Richard Jones <richard@users.sourceforge.net>
parents:
224
diff
changeset
|
108 self.transactions = [] |
|
5096
e74c3611b138
- issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents:
5067
diff
changeset
|
109 self.indexer = get_indexer(config, self) |
|
902
b0d3d3535998
Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
891
diff
changeset
|
110 self.security = security.Security(self) |
|
3609
f2fda3e6fc8b
umask is now configurable (with the same 0002 default)
Richard Jones <richard@users.sourceforge.net>
parents:
3601
diff
changeset
|
111 os.umask(config.UMASK) |
|
440
de5bf4191f11
Enabled transaction support in the bsddb backend.
Richard Jones <richard@users.sourceforge.net>
parents:
431
diff
changeset
|
112 |
|
5041
5251e97b1de0
Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents:
5011
diff
changeset
|
113 # make sure the database directory exists |
|
5251e97b1de0
Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents:
5011
diff
changeset
|
114 if not os.path.isdir(self.config.DATABASE): |
|
5251e97b1de0
Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents:
5011
diff
changeset
|
115 os.makedirs(self.config.DATABASE) |
|
5251e97b1de0
Configure the database backend in the config.ini
John Kristensen <john@jerrykan.com>
parents:
5011
diff
changeset
|
116 |
|
1333
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
117 # lock it |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
118 lockfilenm = os.path.join(self.dir, 'lock') |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
119 self.lockfile = locking.acquire_lock(lockfilenm) |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
120 self.lockfile.write(str(os.getpid())) |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
121 self.lockfile.flush() |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
122 |
|
5319
62de601bdf6f
Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5318
diff
changeset
|
123 self.Session = None |
|
62de601bdf6f
Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5318
diff
changeset
|
124 self.Otk = None |
|
62de601bdf6f
Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5318
diff
changeset
|
125 |
|
825
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
126 def post_init(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
127 """Called once the schema initialisation has finished. |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
128 """ |
|
6148
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
129 super(Database, self).post_init() |
|
825
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
130 # reindex the db if necessary |
|
826
6d7a45c8464a
Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents:
825
diff
changeset
|
131 if self.indexer.should_reindex(): |
|
6d7a45c8464a
Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents:
825
diff
changeset
|
132 self.reindex() |
|
6d7a45c8464a
Added reindex command to roundup-admin.
Richard Jones <richard@users.sourceforge.net>
parents:
825
diff
changeset
|
133 |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1809
diff
changeset
|
134 def refresh_database(self): |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
135 """Rebuild the database |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
136 """ |
|
1840
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1809
diff
changeset
|
137 self.reindex() |
|
91a4619b1a14
hyperdb grows a refresh_database() method.
Anthony Baxter <anthonybaxter@users.sourceforge.net>
parents:
1809
diff
changeset
|
138 |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
139 def getSessionManager(self): |
|
5319
62de601bdf6f
Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5318
diff
changeset
|
140 if not self.Session: |
|
6814
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
141 if self.config.SESSIONDB_BACKEND == "redis": |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
142 if sessions_redis is None: |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
143 self.Session = sessions_dbm.Sessions(self) |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
144 raise ValueError("[redis] session is set, but " |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
145 "redis is not found") |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
146 self.Session = sessions_redis.Sessions(self) |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
147 else: |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
148 self.Session = sessions_dbm.Sessions(self) |
|
5319
62de601bdf6f
Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5318
diff
changeset
|
149 return self.Session |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
150 |
|
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
151 def getOTKManager(self): |
|
5319
62de601bdf6f
Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5318
diff
changeset
|
152 if not self.Otk: |
|
6814
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
153 if self.config.SESSIONDB_BACKEND == "redis": |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
154 if sessions_redis is None: |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
155 self.Session = sessions_dbm.OneTimeKeys(self) |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
156 raise ValueError("[redis] session is set, but " |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
157 "redis is not found") |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
158 self.Otk = sessions_redis.OneTimeKeys(self) |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
159 else: |
|
3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
6658
diff
changeset
|
160 self.Otk = sessions_dbm.OneTimeKeys(self) |
|
5319
62de601bdf6f
Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5318
diff
changeset
|
161 return self.Otk |
|
2082
c091cacdc505
Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
2077
diff
changeset
|
162 |
|
3589
1be293265e61
woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
163 def reindex(self, classname=None, show_progress=False): |
|
2650
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
164 if classname: |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
165 classes = [self.getclass(classname)] |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
166 else: |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
167 classes = self.classes.values() |
|
d68a444fcce3
roundup-admin reindex command may now work on single items or classes
Richard Jones <richard@users.sourceforge.net>
parents:
2649
diff
changeset
|
168 for klass in classes: |
|
3589
1be293265e61
woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
169 if show_progress: |
|
1be293265e61
woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
170 for nodeid in support.Progress('Reindex %s'%klass.classname, |
|
1be293265e61
woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
171 klass.list()): |
|
1be293265e61
woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
172 klass.index(nodeid) |
|
1be293265e61
woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
173 else: |
|
1be293265e61
woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
174 for nodeid in klass.list(): |
|
1be293265e61
woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents:
3586
diff
changeset
|
175 klass.index(nodeid) |
|
825
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
176 self.indexer.save_index() |
|
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
177 |
|
452
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
178 def __repr__(self): |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
179 return '<back_anydbm instance at %x>'%id(self) |
|
452
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
180 |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
181 # |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
182 # Classes |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
183 # |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
184 def __getattr__(self, classname): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
185 """A convenient way of calling self.getclass(classname).""" |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
186 if classname in self.classes: |
|
452
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
187 return self.classes[classname] |
|
5378
35ea9b1efc14
Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5319
diff
changeset
|
188 raise AttributeError(classname) |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
189 |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
190 def addclass(self, cl): |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
191 cn = cl.classname |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
192 if cn in self.classes: |
|
6238
6834bb5473da
Summary: Constrain format of classname and document
John Rouillard <rouilj@ieee.org>
parents:
6209
diff
changeset
|
193 raise ValueError(_('Class "%s" already defined.'%cn)) |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
194 self.classes[cn] = cl |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
195 |
|
2076
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
196 # add default Edit and View permissions |
|
2649
1df7d4a41da4
Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents:
2640
diff
changeset
|
197 self.security.addPermission(name="Create", klass=cn, |
|
1df7d4a41da4
Buncha stuff (sorry about the large checkin):
Richard Jones <richard@users.sourceforge.net>
parents:
2640
diff
changeset
|
198 description="User is allowed to create "+cn) |
|
2076
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
199 self.security.addPermission(name="Edit", klass=cn, |
|
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
200 description="User is allowed to edit "+cn) |
|
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
201 self.security.addPermission(name="View", klass=cn, |
|
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
202 description="User is allowed to access "+cn) |
|
4518
e572c5e28d0b
Fixed typo in permission "Reture" -> "Retire".
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4517
diff
changeset
|
203 self.security.addPermission(name="Retire", klass=cn, |
|
4517
f8e85cf5f0fe
how odd, the Retire permission wasn't being registered;
Richard Jones <richard@users.sourceforge.net>
parents:
4483
diff
changeset
|
204 description="User is allowed to retire "+cn) |
|
2076
2a4309450202
security fixes and doc updates
Richard Jones <richard@users.sourceforge.net>
parents:
2005
diff
changeset
|
205 |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
206 def getclasses(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
207 """Return a list of the names of all existing classes.""" |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
208 return sorted(self.classes.keys()) |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
209 |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
210 def getclass(self, classname): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
211 """Get the Class object representing a particular class. |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
212 |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
213 If 'classname' is not a valid class name, a KeyError is raised. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
214 """ |
|
1145
81941abedb0a
nicer error message for invalid class lookup
Richard Jones <richard@users.sourceforge.net>
parents:
1143
diff
changeset
|
215 try: |
|
81941abedb0a
nicer error message for invalid class lookup
Richard Jones <richard@users.sourceforge.net>
parents:
1143
diff
changeset
|
216 return self.classes[classname] |
|
81941abedb0a
nicer error message for invalid class lookup
Richard Jones <richard@users.sourceforge.net>
parents:
1143
diff
changeset
|
217 except KeyError: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
218 raise KeyError('There is no class called "%s"'%classname) |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
219 |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
220 # |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
221 # Class DBs |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
222 # |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
223 def clear(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
224 """Delete all database contents |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
225 """ |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
226 logging.getLogger('roundup.hyperdb.backend').info('clear') |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
227 for cn in self.classes: |
|
650
9b2575610953
Ran it through pychecker, made fixes
Richard Jones <richard@users.sourceforge.net>
parents:
646
diff
changeset
|
228 for dummy in 'nodes', 'journals': |
|
443
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
229 path = os.path.join(self.dir, 'journals.%s'%cn) |
|
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
230 if os.path.exists(path): |
|
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
231 os.remove(path) |
|
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
232 elif os.path.exists(path+'.db'): # dbm appends .db |
|
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
233 os.remove(path+'.db') |
|
2736
402d6d556558
postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents:
2699
diff
changeset
|
234 # reset id sequences |
|
402d6d556558
postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents:
2699
diff
changeset
|
235 path = os.path.join(os.getcwd(), self.dir, '_ids') |
|
402d6d556558
postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents:
2699
diff
changeset
|
236 if os.path.exists(path): |
|
402d6d556558
postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents:
2699
diff
changeset
|
237 os.remove(path) |
|
402d6d556558
postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents:
2699
diff
changeset
|
238 elif os.path.exists(path+'.db'): # dbm appends .db |
|
402d6d556558
postgres backend open doesn't hide corruption in schema [SF#956375]
Richard Jones <richard@users.sourceforge.net>
parents:
2699
diff
changeset
|
239 os.remove(path+'.db') |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
240 |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
241 def getclassdb(self, classname, mode='r'): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
242 """ grab a connection to the class db that will be used for |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
243 multiple actions |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
244 """ |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
245 return self.opendb('nodes.%s'%classname, mode) |
|
444
3fa2268041a8
Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents:
443
diff
changeset
|
246 |
|
862
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
247 def determine_db_type(self, path): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
248 """ determine which DB wrote the class file |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
249 """ |
|
444
3fa2268041a8
Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents:
443
diff
changeset
|
250 db_type = '' |
|
3fa2268041a8
Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents:
443
diff
changeset
|
251 if os.path.exists(path): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
252 db_type = whichdb(path) |
|
444
3fa2268041a8
Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents:
443
diff
changeset
|
253 if not db_type: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
254 raise hyperdb.DatabaseError(_("Couldn't identify database type")) |
|
444
3fa2268041a8
Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents:
443
diff
changeset
|
255 elif os.path.exists(path+'.db'): |
|
3fa2268041a8
Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents:
443
diff
changeset
|
256 # if the path ends in '.db', it's a dbm database, whether |
|
3fa2268041a8
Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents:
443
diff
changeset
|
257 # anydbm says it's dbhash or not! |
|
3fa2268041a8
Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents:
443
diff
changeset
|
258 db_type = 'dbm' |
|
862
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
259 return db_type |
|
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
260 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
261 def opendb(self, name, mode): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
262 """Low-level database opener that gets around anydbm/dbm |
|
862
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
263 eccentricities. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
264 """ |
|
862
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
265 # figure the class db type |
|
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
266 path = os.path.join(os.getcwd(), self.dir, name) |
|
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
267 db_type = self.determine_db_type(path) |
|
443
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
268 |
|
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
269 # new database? let anydbm pick the best dbm |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
270 # in Python 3+ the "dbm" ("anydbm" to us) module already uses the |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
271 # whichdb() function to do this |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
272 if not db_type or hasattr(anydbm, 'whichdb'): |
|
719
fed4c363a7f3
node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents:
697
diff
changeset
|
273 if __debug__: |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
274 logging.getLogger('roundup.hyperdb.backend').debug( |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
275 "opendb anydbm.open(%r, 'c')"%path) |
|
1028
16498e77e3ff
allow overiding of the index args roundup/cgi/templating.py
Richard Jones <richard@users.sourceforge.net>
parents:
1022
diff
changeset
|
276 return anydbm.open(path, 'c') |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
277 |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
278 # in Python <3 it anydbm was a little dumb so manually open the |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
279 # database with the correct module |
|
443
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
280 try: |
|
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
281 dbm = __import__(db_type) |
|
444
3fa2268041a8
Cor blimey this anydbm/whichdb stuff is yecchy.
Richard Jones <richard@users.sourceforge.net>
parents:
443
diff
changeset
|
282 except ImportError: |
|
6209
e9d12d516517
If gdbm import fails try python3 fallback
John Rouillard <rouilj@ieee.org>
parents:
6179
diff
changeset
|
283 if db_type == 'gdbm': |
|
e9d12d516517
If gdbm import fails try python3 fallback
John Rouillard <rouilj@ieee.org>
parents:
6179
diff
changeset
|
284 try: |
|
e9d12d516517
If gdbm import fails try python3 fallback
John Rouillard <rouilj@ieee.org>
parents:
6179
diff
changeset
|
285 dbm = __import__('dbm.gnu') |
|
e9d12d516517
If gdbm import fails try python3 fallback
John Rouillard <rouilj@ieee.org>
parents:
6179
diff
changeset
|
286 except ImportError: |
|
e9d12d516517
If gdbm import fails try python3 fallback
John Rouillard <rouilj@ieee.org>
parents:
6179
diff
changeset
|
287 raise hyperdb.DatabaseError(_( |
|
e9d12d516517
If gdbm import fails try python3 fallback
John Rouillard <rouilj@ieee.org>
parents:
6179
diff
changeset
|
288 "Couldn't open database - the required module '%s' " |
|
e9d12d516517
If gdbm import fails try python3 fallback
John Rouillard <rouilj@ieee.org>
parents:
6179
diff
changeset
|
289 "(as dbm.gnu) is not available")%db_type) |
|
e9d12d516517
If gdbm import fails try python3 fallback
John Rouillard <rouilj@ieee.org>
parents:
6179
diff
changeset
|
290 else: |
|
e9d12d516517
If gdbm import fails try python3 fallback
John Rouillard <rouilj@ieee.org>
parents:
6179
diff
changeset
|
291 raise hyperdb.DatabaseError(_("Couldn't open database - the " |
|
e9d12d516517
If gdbm import fails try python3 fallback
John Rouillard <rouilj@ieee.org>
parents:
6179
diff
changeset
|
292 "required module '%s' is not available")%db_type) |
|
719
fed4c363a7f3
node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents:
697
diff
changeset
|
293 if __debug__: |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
294 logging.getLogger('roundup.hyperdb.backend').debug( |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
295 "opendb %r.open(%r, %r)"%(db_type, path, mode)) |
|
443
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
296 return dbm.open(path, mode) |
|
a0c598702f17
I fixed the problems with anydbm using the dbm module at the backend.
Richard Jones <richard@users.sourceforge.net>
parents:
440
diff
changeset
|
297 |
|
690
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
298 # |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
299 # Node IDs |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
300 # |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
301 def newid(self, classname): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
302 """ Generate a new id for the given class |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
303 """ |
|
690
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
304 # open the ids DB - create if if doesn't exist |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
305 db = self.opendb('_ids', 'c') |
|
5011
d5da643b3d25
Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
306 if classname in db: |
|
690
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
307 newid = db[classname] = str(int(db[classname]) + 1) |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
308 else: |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
309 # the count() bit is transitional - older dbs won't start at 1 |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
310 newid = str(self.getclass(classname).count()+1) |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
311 db[classname] = newid |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
312 db.close() |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
313 return newid |
|
509a101305da
node ids are now generated from a lockable store - no more race conditions
Richard Jones <richard@users.sourceforge.net>
parents:
676
diff
changeset
|
314 |
|
952
f615fbd02c18
full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents:
950
diff
changeset
|
315 def setid(self, classname, setid): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
316 """ Set the id counter: used during import of database |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
317 """ |
|
952
f615fbd02c18
full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents:
950
diff
changeset
|
318 # open the ids DB - create if if doesn't exist |
|
f615fbd02c18
full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents:
950
diff
changeset
|
319 db = self.opendb('_ids', 'c') |
|
f615fbd02c18
full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents:
950
diff
changeset
|
320 db[classname] = str(setid) |
|
f615fbd02c18
full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents:
950
diff
changeset
|
321 db.close() |
|
f615fbd02c18
full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents:
950
diff
changeset
|
322 |
|
48
671203878652
Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents:
46
diff
changeset
|
323 # |
|
671203878652
Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents:
46
diff
changeset
|
324 # Nodes |
|
671203878652
Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents:
46
diff
changeset
|
325 # |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
326 def addnode(self, classname, nodeid, node): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
327 """ add the specified node to its class's db |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
328 """ |
|
1176
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
329 # we'll be supplied these props if we're doing an import |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
330 if 'creator' not in node: |
|
1176
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
331 # add in the "calculated" properties (dupe so we don't affect |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
332 # calling code's node assumptions) |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
333 node = node.copy() |
|
1800
a3b1b1dcf639
Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
1794
diff
changeset
|
334 node['creator'] = self.getuid() |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
335 node['actor'] = self.getuid() |
|
1176
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
336 node['creation'] = node['activity'] = date.Date() |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
337 |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
338 self.newnodes.setdefault(classname, {})[nodeid] = 1 |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
339 self.cache.setdefault(classname, {})[nodeid] = node |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
340 self.savenode(classname, nodeid, node) |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
341 |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
342 def setnode(self, classname, nodeid, node): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
343 """ change the specified node |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
344 """ |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
345 self.dirtynodes.setdefault(classname, {})[nodeid] = 1 |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
346 |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
347 # can't set without having already loaded the node |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
348 self.cache[classname][nodeid] = node |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
349 self.savenode(classname, nodeid, node) |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
350 |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
351 def savenode(self, classname, nodeid, node): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
352 """ perform the saving of data specified by the set/addnode |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
353 """ |
|
719
fed4c363a7f3
node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents:
697
diff
changeset
|
354 if __debug__: |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
355 logging.getLogger('roundup.hyperdb.backend').debug( |
|
4421
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
356 'save %s%s %r'%(classname, nodeid, node)) |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
357 self.transactions.append((self.doSaveNode, (classname, nodeid, node))) |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
358 |
|
475
a1a44636bace
Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents:
464
diff
changeset
|
359 def getnode(self, classname, nodeid, db=None, cache=1): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
360 """ get a node from the database |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
361 |
|
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
362 Note the "cache" parameter is not used, and exists purely for |
|
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
363 backward compatibility! |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
364 """ |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
365 # try the cache |
|
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
366 cache_dict = self.cache.setdefault(classname, {}) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
367 if nodeid in cache_dict: |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
368 if __debug__: |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
369 logging.getLogger('roundup.hyperdb.backend').debug( |
|
4421
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
370 'get %s%s cached'%(classname, nodeid)) |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
371 self.stats['cache_hits'] += 1 |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
372 return cache_dict[nodeid] |
|
719
fed4c363a7f3
node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents:
697
diff
changeset
|
373 |
|
fed4c363a7f3
node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents:
697
diff
changeset
|
374 if __debug__: |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
375 self.stats['cache_misses'] += 1 |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
376 start_t = time.time() |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
377 logging.getLogger('roundup.hyperdb.backend').debug( |
|
4421
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
378 'get %s%s'%(classname, nodeid)) |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
379 |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
380 # get from the database and save in the cache |
|
452
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
381 if db is None: |
|
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
382 db = self.getclassdb(classname) |
|
5011
d5da643b3d25
Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
383 if nodeid not in db: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
384 raise IndexError("no such %s %s"%(classname, nodeid)) |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
385 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
386 # check the uncommitted, destroyed nodes |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
387 if (classname in self.destroyednodes and |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
388 nodeid in self.destroyednodes[classname]): |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
389 raise IndexError("no such %s %s"%(classname, nodeid)) |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
390 |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
391 # decode |
|
48
671203878652
Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents:
46
diff
changeset
|
392 res = marshal.loads(db[nodeid]) |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
393 |
|
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
394 # reverse the serialisation |
|
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
395 res = self.unserialise(classname, res) |
|
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
396 |
|
719
fed4c363a7f3
node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents:
697
diff
changeset
|
397 # store off in the cache dict |
|
475
a1a44636bace
Fix breakage caused by transaction changes.
Richard Jones <richard@users.sourceforge.net>
parents:
464
diff
changeset
|
398 if cache: |
|
719
fed4c363a7f3
node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents:
697
diff
changeset
|
399 cache_dict[nodeid] = res |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
400 |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
401 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
402 self.stats['get_items'] += (time.time() - start_t) |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
403 |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
404 return res |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
405 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
406 def destroynode(self, classname, nodeid): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
407 """Remove a node from the database. Called exclusively by the |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
408 destroy() method on Class. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
409 """ |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
410 logging.getLogger('roundup.hyperdb.backend').info( |
|
4421
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
411 'destroy %s%s'%(classname, nodeid)) |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
412 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
413 # remove from cache and newnodes if it's there |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
414 if (classname in self.cache and nodeid in self.cache[classname]): |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
415 del self.cache[classname][nodeid] |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
416 if (classname in self.newnodes and nodeid in self.newnodes[classname]): |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
417 del self.newnodes[classname][nodeid] |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
418 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
419 # see if there's any obvious commit actions that we should get rid of |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
420 for entry in self.transactions[:]: |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
421 if entry[1][:2] == (classname, nodeid): |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
422 self.transactions.remove(entry) |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
423 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
424 # add to the destroyednodes map |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
425 self.destroyednodes.setdefault(classname, {})[nodeid] = 1 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
426 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
427 # add the destroy commit action |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
428 self.transactions.append((self.doDestroyNode, (classname, nodeid))) |
|
3906
666b70676ec6
destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3897
diff
changeset
|
429 self.transactions.append((FileStorage.destroy, (self, classname, nodeid))) |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
430 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
431 def serialise(self, classname, node): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
432 """Copy the node contents, converting non-marshallable data into |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
433 marshallable data. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
434 """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
435 properties = self.getclass(classname).getprops() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
436 d = {} |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
437 for k, v in node.items(): |
|
2964
b71f6d0a463d
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2908
diff
changeset
|
438 if k == self.RETIRED_FLAG: |
|
b71f6d0a463d
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2908
diff
changeset
|
439 d[k] = v |
|
b71f6d0a463d
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2908
diff
changeset
|
440 continue |
|
b71f6d0a463d
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2908
diff
changeset
|
441 |
|
b71f6d0a463d
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2908
diff
changeset
|
442 # if the property doesn't exist then we really don't care |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
443 if k not in properties: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
444 continue |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
445 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
446 # get the property spec |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
447 prop = properties[k] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
448 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
449 if isinstance(prop, hyperdb.Password) and v is not None: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
450 d[k] = str(v) |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
451 elif isinstance(prop, hyperdb.Date) and v is not None: |
|
964
832d1209aaa2
Preparing to turn back on link/unlink journal events.
Richard Jones <richard@users.sourceforge.net>
parents:
952
diff
changeset
|
452 d[k] = v.serialise() |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
453 elif isinstance(prop, hyperdb.Interval) and v is not None: |
|
964
832d1209aaa2
Preparing to turn back on link/unlink journal events.
Richard Jones <richard@users.sourceforge.net>
parents:
952
diff
changeset
|
454 d[k] = v.serialise() |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
455 else: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
456 d[k] = v |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
457 return d |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
458 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
459 def unserialise(self, classname, node): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
460 """Decode the marshalled node data |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
461 """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
462 properties = self.getclass(classname).getprops() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
463 d = {} |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
464 for k, v in node.items(): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
465 # if the property doesn't exist, or is the "retired" flag then |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
466 # it won't be in the properties dict |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
467 if k not in properties: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
468 d[k] = v |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
469 continue |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
470 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
471 # get the property spec |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
472 prop = properties[k] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
473 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
474 if isinstance(prop, hyperdb.Date) and v is not None: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
475 d[k] = date.Date(v) |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
476 elif isinstance(prop, hyperdb.Interval) and v is not None: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
477 d[k] = date.Interval(v) |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
478 elif isinstance(prop, hyperdb.Password) and v is not None: |
|
4480
1613754d2646
Fix first part of Password handling security issue2550688
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
479 d[k] = password.Password(encrypted=v) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
480 else: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
481 d[k] = v |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
482 return d |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
483 |
|
452
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
484 def hasnode(self, classname, nodeid, db=None): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
485 """ determine if the database has a given node |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
486 """ |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
487 # try the cache |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
488 cache = self.cache.setdefault(classname, {}) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
489 if nodeid in cache: |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
490 return 1 |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
491 |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
492 # not in the cache - check the database |
|
452
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
493 if db is None: |
|
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
494 db = self.getclassdb(classname) |
|
5011
d5da643b3d25
Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
495 return nodeid in db |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
496 |
|
452
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
497 def countnodes(self, classname, db=None): |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
498 count = 0 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
499 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
500 # include the uncommitted nodes |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
501 if classname in self.newnodes: |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
502 count += len(self.newnodes[classname]) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
503 if classname in self.destroyednodes: |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
504 count -= len(self.destroyednodes[classname]) |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
505 |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
506 # and count those in the DB |
|
452
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
507 if db is None: |
|
7181efddce66
yuck, a gdbm instance tests false :(
Richard Jones <richard@users.sourceforge.net>
parents:
444
diff
changeset
|
508 db = self.getclassdb(classname) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
509 return count + len(db) |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
510 |
|
461
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
511 |
|
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
512 # |
|
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
513 # Files - special node properties |
|
646
07abfe8f0c01
use blobfiles in back_anydbm which is used in back_bsddb.
Engelbert Gruber <grubert@users.sourceforge.net>
parents:
624
diff
changeset
|
514 # inherited from FileStorage |
|
461
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
515 |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
516 # |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
517 # Journal |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
518 # |
| 1143 | 519 def addjournal(self, classname, nodeid, action, params, creator=None, |
| 520 creation=None): | |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
521 """ Journal the Action |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
522 'action' may be: |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
523 |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
524 'set' -- 'params' is a dictionary of property values |
|
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
525 'create' -- 'params' is an empty dictionary as of |
|
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
526 Wed Nov 06 11:38:43 2002 +0000 |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
527 'link' or 'unlink' -- 'params' is (classname, nodeid, propname) |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
528 'retired' or 'restored' -- 'params' is None |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
529 |
|
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
530 'creator' -- the user performing the action, which defaults to |
|
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
531 the current user. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
532 """ |
|
719
fed4c363a7f3
node caching now works, and gives a small boost in performance
Richard Jones <richard@users.sourceforge.net>
parents:
697
diff
changeset
|
533 if __debug__: |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
534 logging.getLogger('roundup.hyperdb.backend').debug( |
|
4421
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
535 'addjournal %s%s %s %r %s %r'%(classname, |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
536 nodeid, action, params, creator, creation)) |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
537 if creator is None: |
|
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
538 creator = self.getuid() |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
539 self.transactions.append((self.doSaveJournal, (classname, nodeid, |
| 1143 | 540 action, params, creator, creation))) |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
541 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
542 def setjournal(self, classname, nodeid, journal): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
543 """Set the journal to the "journal" list.""" |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
544 if __debug__: |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
545 logging.getLogger('roundup.hyperdb.backend').debug( |
|
4421
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
546 'setjournal %s%s %r'%(classname, nodeid, journal)) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
547 self.transactions.append((self.doSetJournal, (classname, nodeid, |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
548 journal))) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
549 |
|
4534
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
550 def fix_journal(self, classname, journal): |
|
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
551 """ fix password entries to correct type """ |
|
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
552 pwprops = {} |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
553 for pn, prop in self.getclass(classname).properties.items(): |
|
4534
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
554 if isinstance(prop, hyperdb.Password): |
|
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
555 pwprops [pn] = 1 |
|
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
556 if not pwprops: |
|
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
557 return journal |
|
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
558 for j in journal: |
|
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
559 if j[3] == 'set': |
|
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
560 for k, v in j[4].items(): |
|
4538
fd972e18b21a
- fix import/export regression test for anydbm for latest journal fix
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4534
diff
changeset
|
561 if k in pwprops and j[4][k]: |
|
4534
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
562 j[4][k] = password.JournalPassword(j[4][k]) |
|
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
563 return journal |
|
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
564 |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
565 def getjournal(self, classname, nodeid): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
566 """ get the journal for id |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
567 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
568 Raise IndexError if the node doesn't exist (as per history()'s |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
569 API) |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
570 """ |
|
1567
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
571 # our journal result |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
572 res = [] |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
573 |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
574 # add any journal entries for transactions not committed to the |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
575 # database |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
576 for method, args in self.transactions: |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
577 if method != self.doSaveJournal: |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
578 continue |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
579 (cache_classname, cache_nodeid, cache_action, cache_params, |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
580 cache_creator, cache_creation) = args |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
581 if cache_classname == classname and cache_nodeid == nodeid: |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
582 if not cache_creator: |
|
1800
a3b1b1dcf639
Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
1794
diff
changeset
|
583 cache_creator = self.getuid() |
|
1567
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
584 if not cache_creation: |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
585 cache_creation = date.Date() |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
586 res.append((cache_nodeid, cache_creation, cache_creator, |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
587 cache_action, cache_params)) |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
588 |
|
48
671203878652
Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents:
46
diff
changeset
|
589 # attempt to open the journal - in some rare cases, the journal may |
|
671203878652
Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents:
46
diff
changeset
|
590 # not exist |
|
671203878652
Moved over to using marshal in the bsddb and anydbm backends.
Richard Jones <richard@users.sourceforge.net>
parents:
46
diff
changeset
|
591 try: |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
592 db = self.opendb('journals.%s'%classname, 'r') |
|
5248
198b6e810c67
Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents:
5232
diff
changeset
|
593 except anydbm.error as error: |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
594 if str(error) == "need 'c' or 'n' flag to open new db": |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
595 raise IndexError('no such %s %s'%(classname, nodeid)) |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
596 elif error.args[0] != 2: |
|
1920
f9316d2cd5ba
Fixed retirement of items in rdbms imports [SF#841355]
Richard Jones <richard@users.sourceforge.net>
parents:
1904
diff
changeset
|
597 # this isn't a "not found" error, be alarmed! |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
598 raise |
|
1920
f9316d2cd5ba
Fixed retirement of items in rdbms imports [SF#841355]
Richard Jones <richard@users.sourceforge.net>
parents:
1904
diff
changeset
|
599 if res: |
|
f9316d2cd5ba
Fixed retirement of items in rdbms imports [SF#841355]
Richard Jones <richard@users.sourceforge.net>
parents:
1904
diff
changeset
|
600 # we have unsaved journal entries, return them |
|
4534
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
601 return self.fix_journal (classname, res) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
602 raise IndexError('no such %s %s'%(classname, nodeid)) |
|
787
b6b0a92e0738
More informative error message
Richard Jones <richard@users.sourceforge.net>
parents:
778
diff
changeset
|
603 try: |
|
b6b0a92e0738
More informative error message
Richard Jones <richard@users.sourceforge.net>
parents:
778
diff
changeset
|
604 journal = marshal.loads(db[nodeid]) |
|
b6b0a92e0738
More informative error message
Richard Jones <richard@users.sourceforge.net>
parents:
778
diff
changeset
|
605 except KeyError: |
|
843
01f9d02faea1
...except of course it's nice to use valid Python syntax
Richard Jones <richard@users.sourceforge.net>
parents:
842
diff
changeset
|
606 db.close() |
|
1567
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
607 if res: |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
608 # we have some unsaved journal entries, be happy! |
|
4534
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
609 return self.fix_journal (classname, res) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
610 raise IndexError('no such %s %s'%(classname, nodeid)) |
|
843
01f9d02faea1
...except of course it's nice to use valid Python syntax
Richard Jones <richard@users.sourceforge.net>
parents:
842
diff
changeset
|
611 db.close() |
|
1567
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
612 |
|
fc8998ce6274
fixed missing (pre-commit) journal entries in *dbm backends [SF#679217]
Richard Jones <richard@users.sourceforge.net>
parents:
1563
diff
changeset
|
613 # add all the saved journal entries for this node |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
614 for nodeid, date_stamp, user, action, params in journal: |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
615 res.append((nodeid, date.Date(date_stamp), user, action, params)) |
|
4534
0dd6bdeb2eb5
issue2550729: Fix password history display for anydbm backend...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4518
diff
changeset
|
616 return self.fix_journal (classname, res) |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
617 |
|
562
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
618 def pack(self, pack_before): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
619 """ Delete all journal entries except "create" before 'pack_before'. |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
620 """ |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1195
diff
changeset
|
621 pack_before = pack_before.serialise() |
| 990 | 622 for classname in self.getclasses(): |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
623 packed = 0 |
| 990 | 624 # get the journal db |
|
562
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
625 db_name = 'journals.%s'%classname |
|
862
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
626 path = os.path.join(os.getcwd(), self.dir, classname) |
|
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
627 db_type = self.determine_db_type(path) |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
628 db = self.opendb(db_name, 'w') |
|
562
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
629 |
|
5444
167f0d25ea8e
Python 3 preparation: convert dbm keys back from bytes to strings.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5395
diff
changeset
|
630 for key in map(b2s, db.keys()): |
| 990 | 631 # get the journal for this db entry |
|
562
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
632 journal = marshal.loads(db[key]) |
|
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
633 l = [] |
|
567
9ca63f7332a7
last_set_entry was referenced before assignment
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
566
diff
changeset
|
634 last_set_entry = None |
|
562
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
635 for entry in journal: |
| 990 | 636 # unpack the entry |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
637 (nodeid, date_stamp, self.journaltag, action, |
|
562
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
638 params) = entry |
| 990 | 639 # if the entry is after the pack date, _or_ the initial |
| 640 # create entry, then it stays | |
|
562
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
641 if date_stamp > pack_before or action == 'create': |
|
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
642 l.append(entry) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
643 else: |
|
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
644 packed += 1 |
|
562
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
645 db[key] = marshal.dumps(l) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
646 |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
647 logging.getLogger('roundup.hyperdb.backend').info( |
|
4421
67bef70ab9b9
- more logger fixes, sorry for the noise.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4420
diff
changeset
|
648 'packed %d %s items'%(packed, classname)) |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
649 |
|
562
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
650 if db_type == 'gdbm': |
|
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
651 db.reorganize() |
|
62febbd7ffec
You can now use the roundup-admin tool to pack the database
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
553
diff
changeset
|
652 db.close() |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
653 |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
654 |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
655 # |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
656 # Basic transaction support |
|
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
657 # |
|
5319
62de601bdf6f
Fix commits although a Reject exception is raised
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5318
diff
changeset
|
658 def commit(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
659 """ Commit the current transactions. |
|
3687
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3682
diff
changeset
|
660 |
|
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3682
diff
changeset
|
661 Save all data changed since the database was opened or since the |
|
ff9f4ca42454
Postgres backend allows transaction collisions to be ignored when...
Richard Jones <richard@users.sourceforge.net>
parents:
3682
diff
changeset
|
662 last commit() or rollback(). |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
663 """ |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
664 logging.getLogger('roundup.hyperdb.backend').info('commit %s transactions'%( |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
665 len(self.transactions))) |
|
461
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
666 |
|
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
667 # keep a handle to all the database files opened |
|
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
668 self.databases = {} |
|
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
669 |
|
1904
9445caec3ff5
more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
670 try: |
|
9445caec3ff5
more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
671 # now, do all the transactions |
|
9445caec3ff5
more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
672 reindex = {} |
|
9445caec3ff5
more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
673 for method, args in self.transactions: |
|
9445caec3ff5
more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
674 reindex[method(*args)] = 1 |
|
9445caec3ff5
more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
675 finally: |
|
9445caec3ff5
more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
676 # make sure we close all the database files |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
677 for db in self.databases.values(): |
|
1904
9445caec3ff5
more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
678 db.close() |
|
9445caec3ff5
more database closing cleanups, finally mysql has no dangling references
Richard Jones <richard@users.sourceforge.net>
parents:
1840
diff
changeset
|
679 del self.databases |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
680 |
|
3960
bc412bb2ccd3
Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents:
3955
diff
changeset
|
681 # clear the transactions list now so the blobfile implementation |
|
bc412bb2ccd3
Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents:
3955
diff
changeset
|
682 # doesn't think there's still pending file commits when it tries |
|
bc412bb2ccd3
Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents:
3955
diff
changeset
|
683 # to access the file data |
|
bc412bb2ccd3
Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents:
3955
diff
changeset
|
684 self.transactions = [] |
|
bc412bb2ccd3
Workaround race condition in file storage during transaction commit [SF#883580]
Richard Jones <richard@users.sourceforge.net>
parents:
3955
diff
changeset
|
685 |
|
825
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
686 # reindex the nodes that request it |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
687 for classname, nodeid in [k for k in reindex if k]: |
|
825
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
688 self.getclass(classname).index(nodeid) |
|
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
689 |
|
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
690 # save the indexer state |
|
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
691 self.indexer.save_index() |
|
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
692 |
|
1181
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
693 self.clearCache() |
|
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
694 |
|
49aebf5a8691
some speedups, some fixes to the benchmarking
Richard Jones <richard@users.sourceforge.net>
parents:
1176
diff
changeset
|
695 def clearCache(self): |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
696 # all transactions committed, back to normal |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
697 self.cache = {} |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
698 self.dirtynodes = {} |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
699 self.newnodes = {} |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
700 self.destroyednodes = {} |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
701 self.transactions = [] |
|
4652
dfbc0cfa9811
Add an interface to register clearCache callbacks in roundupdb.
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4538
diff
changeset
|
702 # upcall is necessary! |
|
dfbc0cfa9811
Add an interface to register clearCache callbacks in roundupdb.
Ralf Schlatterbeck <rsc@runtux.com>
parents:
4538
diff
changeset
|
703 roundupdb.Database.clearCache(self) |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
704 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
705 def getCachedClassDB(self, classname): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
706 """ get the class db, looking in our cache of databases for commit |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
707 """ |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
708 # get the database handle |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
709 db_name = 'nodes.%s'%classname |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
710 if db_name not in self.databases: |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
711 self.databases[db_name] = self.getclassdb(classname, 'c') |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
712 return self.databases[db_name] |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
713 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
714 def doSaveNode(self, classname, nodeid, node): |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
715 db = self.getCachedClassDB(classname) |
|
461
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
716 |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
717 # now save the marshalled data |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
718 db[nodeid] = marshal.dumps(self.serialise(classname, node)) |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
719 |
|
825
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
720 # return the classname, nodeid so we reindex this content |
|
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
721 return (classname, nodeid) |
|
0779ea9f1f18
More indexer work:
Richard Jones <richard@users.sourceforge.net>
parents:
818
diff
changeset
|
722 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
723 def getCachedJournalDB(self, classname): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
724 """ get the journal db, looking in our cache of databases for commit |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
725 """ |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
726 # get the database handle |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
727 db_name = 'journals.%s'%classname |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
728 if db_name not in self.databases: |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
729 self.databases[db_name] = self.opendb(db_name, 'c') |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
730 return self.databases[db_name] |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
731 |
| 1143 | 732 def doSaveJournal(self, classname, nodeid, action, params, creator, |
| 733 creation): | |
| 734 # serialise the parameters now if necessary | |
| 735 if isinstance(params, type({})): | |
| 736 if action in ('set', 'create'): | |
| 737 params = self.serialise(classname, params) | |
| 738 | |
|
952
f615fbd02c18
full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents:
950
diff
changeset
|
739 # handle supply of the special journalling parameters (usually |
|
f615fbd02c18
full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents:
950
diff
changeset
|
740 # supplied on importing an existing database) |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
741 journaltag = creator |
| 1143 | 742 if creation: |
| 743 journaldate = creation.serialise() | |
| 744 else: | |
|
964
832d1209aaa2
Preparing to turn back on link/unlink journal events.
Richard Jones <richard@users.sourceforge.net>
parents:
952
diff
changeset
|
745 journaldate = date.Date().serialise() |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
746 |
|
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
747 # create the journal entry |
|
952
f615fbd02c18
full database export and import is done
Richard Jones <richard@users.sourceforge.net>
parents:
950
diff
changeset
|
748 entry = (nodeid, journaldate, journaltag, action, params) |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
749 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
750 db = self.getCachedJournalDB(classname) |
|
461
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
751 |
|
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
752 # now insert the journal entry |
|
5011
d5da643b3d25
Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
753 if nodeid in db: |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
754 # append to existing |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
755 s = db[nodeid] |
|
650
9b2575610953
Ran it through pychecker, made fixes
Richard Jones <richard@users.sourceforge.net>
parents:
646
diff
changeset
|
756 l = marshal.loads(s) |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
757 l.append(entry) |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
758 else: |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
759 l = [entry] |
|
676
bc46480e2a2b
Fixed serialisation problem...
Richard Jones <richard@users.sourceforge.net>
parents:
650
diff
changeset
|
760 |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
761 db[nodeid] = marshal.dumps(l) |
|
461
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
762 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
763 def doSetJournal(self, classname, nodeid, journal): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
764 l = [] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
765 for nodeid, journaldate, journaltag, action, params in journal: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
766 # serialise the parameters now if necessary |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
767 if isinstance(params, type({})): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
768 if action in ('set', 'create'): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
769 params = self.serialise(classname, params) |
|
2512
f5542d92307a
fix anydbm journal import [SF#983166]
Richard Jones <richard@users.sourceforge.net>
parents:
2505
diff
changeset
|
770 journaldate = journaldate.serialise() |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
771 l.append((nodeid, journaldate, journaltag, action, params)) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
772 db = self.getCachedJournalDB(classname) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
773 db[nodeid] = marshal.dumps(l) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
774 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
775 def doDestroyNode(self, classname, nodeid): |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
776 # delete from the class database |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
777 db = self.getCachedClassDB(classname) |
|
5011
d5da643b3d25
Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
778 if nodeid in db: |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
779 del db[nodeid] |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
780 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
781 # delete from the database |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
782 db = self.getCachedJournalDB(classname) |
|
5011
d5da643b3d25
Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
783 if nodeid in db: |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
784 del db[nodeid] |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
785 |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
786 def rollback(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
787 """ Reverse all actions from the current transaction. |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
788 """ |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
789 logging.getLogger('roundup.hyperdb.backend').info('rollback %s transactions'%( |
|
2514
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
790 len(self.transactions))) |
|
091711fb2f8c
Initial logging integration: replace all debug prints with logging calls...
Richard Jones <richard@users.sourceforge.net>
parents:
2512
diff
changeset
|
791 |
|
461
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
792 for method, args in self.transactions: |
|
b579418f7ed1
Implemented file store rollback.
Richard Jones <richard@users.sourceforge.net>
parents:
460
diff
changeset
|
793 # delete temporary files |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
794 if method == self.doStoreFile: |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
795 self.rollbackStoreFile(*args) |
|
430
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
796 self.cache = {} |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
797 self.dirtynodes = {} |
|
350685601f37
Database transactions.
Richard Jones <richard@users.sourceforge.net>
parents:
394
diff
changeset
|
798 self.newnodes = {} |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
799 self.destroyednodes = {} |
|
252
76c6994aa4e8
CGI interfaces now spit up a top-level index of all instances they can serve.
Richard Jones <richard@users.sourceforge.net>
parents:
224
diff
changeset
|
800 self.transactions = [] |
|
46
3c5920433866
*sigh* some databases have _foo.so as their underlying implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
801 |
|
1131
92e92ae58494
add close() methods where they are missing!
Richard Jones <richard@users.sourceforge.net>
parents:
1127
diff
changeset
|
802 def close(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
803 """ Nothing to do |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
804 """ |
|
1333
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
805 if self.lockfile is not None: |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
806 locking.release_lock(self.lockfile) |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
807 self.lockfile.close() |
|
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
808 self.lockfile = None |
|
1131
92e92ae58494
add close() methods where they are missing!
Richard Jones <richard@users.sourceforge.net>
parents:
1127
diff
changeset
|
809 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
810 _marker = [] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
811 class Class(hyperdb.Class): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
812 """The handle to a particular class of nodes in a hyperdatabase.""" |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
813 |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
814 def enableJournalling(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
815 """Turn journalling on for this class |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
816 """ |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
817 self.do_journal = 1 |
|
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
818 |
|
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
819 def disableJournalling(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
820 """Turn journalling off for this class |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
821 """ |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
822 self.do_journal = 0 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
823 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
824 # Editing nodes: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
825 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
826 def create(self, **propvalues): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
827 """Create a new node of this class and return its id. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
828 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
829 The keyword arguments in 'propvalues' map property names to values. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
830 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
831 The values of arguments must be acceptable for the types of their |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
832 corresponding properties or a TypeError is raised. |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
833 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
834 If this class has a key property, it must be present and its value |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
835 must not collide with other key strings or a ValueError is raised. |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
836 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
837 Any other properties on this class that are missing from the |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
838 'propvalues' dictionary are set to None. |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
839 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
840 If an id in a link or multilink property does not refer to a valid |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
841 node, an IndexError is raised. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
842 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
843 These operations trigger detectors and can be vetoed. Attempts |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
844 to modify the "creation" or "activity" properties cause a KeyError. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
845 """ |
|
4347
0e33bf5571dc
make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents:
4342
diff
changeset
|
846 if self.db.journaltag is None: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
847 raise hyperdb.DatabaseError(_('Database open read-only')) |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
848 self.fireAuditors('create', None, propvalues) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
849 newid = self.create_inner(**propvalues) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
850 self.fireReactors('create', newid, None) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
851 return newid |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
852 |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
853 def create_inner(self, **propvalues): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
854 """ Called by create, in-between the audit and react calls. |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
855 """ |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
856 if 'id' in propvalues: |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
857 raise KeyError('"id" is reserved') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
858 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
859 if self.db.journaltag is None: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
860 raise hyperdb.DatabaseError(_('Database open read-only')) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
861 |
|
5704
aa13a517cc63
Anydbm back end didn't list creator or actor as protected properties.
John Rouillard <rouilj@ieee.org>
parents:
5544
diff
changeset
|
862 |
|
aa13a517cc63
Anydbm back end didn't list creator or actor as protected properties.
John Rouillard <rouilj@ieee.org>
parents:
5544
diff
changeset
|
863 if ('creator' in propvalues or 'actor' in propvalues or |
|
aa13a517cc63
Anydbm back end didn't list creator or actor as protected properties.
John Rouillard <rouilj@ieee.org>
parents:
5544
diff
changeset
|
864 'creation' in propvalues or 'activity' in propvalues): |
|
aa13a517cc63
Anydbm back end didn't list creator or actor as protected properties.
John Rouillard <rouilj@ieee.org>
parents:
5544
diff
changeset
|
865 raise KeyError('"creator", "actor", "creation" and ' |
|
aa13a517cc63
Anydbm back end didn't list creator or actor as protected properties.
John Rouillard <rouilj@ieee.org>
parents:
5544
diff
changeset
|
866 '"activity" are reserved') |
|
6148
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
867 |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
868 for p in propvalues: |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
869 prop = self.properties[p] |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
870 if prop.computed: |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
871 raise KeyError('"%s" is a computed property'%p) |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
872 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
873 # new node's id |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
874 newid = self.db.newid(self.classname) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
875 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
876 # validate propvalues |
| 5809 | 877 num_re = re.compile(r'^\d+$') |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
878 for key, value in propvalues.items(): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
879 if key == self.key: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
880 try: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
881 self.lookup(value) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
882 except KeyError: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
883 pass |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
884 else: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
885 raise ValueError('node with key "%s" exists'%value) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
886 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
887 # try to handle this property |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
888 try: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
889 prop = self.properties[key] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
890 except KeyError: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
891 raise KeyError('"%s" has no property "%s"'%(self.classname, |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
892 key)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
893 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
894 if value is not None and isinstance(prop, hyperdb.Link): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
895 if type(value) != type(''): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
896 raise ValueError('link value must be String') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
897 link_class = self.properties[key].classname |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
898 # if it isn't a number, it's a key |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
899 if not num_re.match(value): |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
900 try: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
901 value = self.db.classes[link_class].lookup(value) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
902 except (TypeError, KeyError): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
903 raise IndexError('new property "%s": %s not a %s'%( |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
904 key, value, link_class)) |
|
902
b0d3d3535998
Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
891
diff
changeset
|
905 elif not self.db.getclass(link_class).hasnode(value): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
906 raise IndexError('%s has no node %s'%(link_class, |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
907 value)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
908 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
909 # save off the value |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
910 propvalues[key] = value |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
911 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
912 # register the link with the newly linked node |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
913 if self.do_journal and self.properties[key].do_journal: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
914 self.db.addjournal(link_class, value, 'link', |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
915 (self.classname, newid, key)) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
916 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
917 elif isinstance(prop, hyperdb.Multilink): |
|
3872
34128a809e22
Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3869
diff
changeset
|
918 if value is None: |
|
34128a809e22
Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3869
diff
changeset
|
919 value = [] |
|
5467
630a00b20394
strings have an __iter__ method on Python 3, but we don't want to consider them iterable
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5444
diff
changeset
|
920 if not hasattr(value, '__iter__') or type(value) == type(''): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
921 raise TypeError('new property "%s" not an iterable of ids'%key) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
922 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
923 # clean up and validate the list of links |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
924 link_class = self.properties[key].classname |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
925 l = [] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
926 for entry in value: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
927 if type(entry) != type(''): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
928 raise ValueError('"%s" multilink value (%r) '\ |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
929 'must contain Strings'%(key, value)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
930 # if it isn't a number, it's a key |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
931 if not num_re.match(entry): |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
932 try: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
933 entry = self.db.classes[link_class].lookup(entry) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
934 except (TypeError, KeyError): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
935 raise IndexError('new property "%s": %s not a %s'%( |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
936 key, entry, self.properties[key].classname)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
937 l.append(entry) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
938 value = l |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
939 propvalues[key] = value |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
940 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
941 # handle additions |
|
902
b0d3d3535998
Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
891
diff
changeset
|
942 for nodeid in value: |
|
b0d3d3535998
Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
891
diff
changeset
|
943 if not self.db.getclass(link_class).hasnode(nodeid): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
944 raise IndexError('%s has no node %s'%(link_class, |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
945 nodeid)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
946 # register the link with the newly linked node |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
947 if self.do_journal and self.properties[key].do_journal: |
|
902
b0d3d3535998
Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
891
diff
changeset
|
948 self.db.addjournal(link_class, nodeid, 'link', |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
949 (self.classname, newid, key)) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
950 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
951 elif isinstance(prop, hyperdb.String): |
|
1383
f19dde90e473
applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1351
diff
changeset
|
952 if type(value) != type('') and type(value) != type(u''): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
953 raise TypeError('new property "%s" not a string'%key) |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2736
diff
changeset
|
954 if prop.indexme: |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2736
diff
changeset
|
955 self.db.indexer.add_text((self.classname, newid, key), |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2736
diff
changeset
|
956 value) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
957 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
958 elif isinstance(prop, hyperdb.Password): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
959 if not isinstance(value, password.Password): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
960 raise TypeError('new property "%s" not a Password'%key) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
961 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
962 elif isinstance(prop, hyperdb.Date): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
963 if value is not None and not isinstance(value, date.Date): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
964 raise TypeError('new property "%s" not a Date'%key) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
965 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
966 elif isinstance(prop, hyperdb.Interval): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
967 if value is not None and not isinstance(value, date.Interval): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
968 raise TypeError('new property "%s" not an Interval'%key) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
969 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
970 elif value is not None and isinstance(prop, hyperdb.Number): |
|
880
de3da99a7c02
Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents:
869
diff
changeset
|
971 try: |
|
890
a568596dbea7
Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
887
diff
changeset
|
972 float(value) |
|
887
e7169d6e6e45
added tests for number type too
Richard Jones <richard@users.sourceforge.net>
parents:
886
diff
changeset
|
973 except ValueError: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
974 raise TypeError('new property "%s" not numeric'%key) |
|
880
de3da99a7c02
Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents:
869
diff
changeset
|
975 |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
976 elif value is not None and isinstance(prop, hyperdb.Integer): |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
977 try: |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
978 int(value) |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
979 except ValueError: |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
980 raise TypeError('new property "%s" not an integer'%key) |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
981 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
982 elif value is not None and isinstance(prop, hyperdb.Boolean): |
|
890
a568596dbea7
Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
887
diff
changeset
|
983 try: |
|
a568596dbea7
Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
887
diff
changeset
|
984 int(value) |
|
a568596dbea7
Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
887
diff
changeset
|
985 except ValueError: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
986 raise TypeError('new property "%s" not boolean'%key) |
|
880
de3da99a7c02
Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents:
869
diff
changeset
|
987 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
988 # make sure there's data where there needs to be |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
989 for key, prop in self.properties.items(): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
990 if key in propvalues: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
991 continue |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
992 if key == self.key: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
993 raise ValueError('key property "%s" is required'%key) |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
994 if isinstance(prop, hyperdb.Multilink): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
995 propvalues[key] = [] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
996 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
997 # done |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
998 self.db.addnode(self.classname, newid, propvalues) |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
999 if self.do_journal: |
|
1304
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1303
diff
changeset
|
1000 self.db.addjournal(self.classname, newid, 'create', {}) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1001 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1002 return newid |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1003 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1004 def get(self, nodeid, propname, default=_marker, cache=1): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1005 """Get the value of a property on an existing node of this class. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1006 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1007 'nodeid' must be the id of an existing node of this class or an |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1008 IndexError is raised. 'propname' must be the name of a property |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1009 of this class or a KeyError is raised. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1010 |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
1011 'cache' exists for backward compatibility, and is not used. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1012 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1013 Attempts to get the "creation" or "activity" properties should |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1014 do the right thing. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1015 """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1016 if propname == 'id': |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1017 return nodeid |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1018 |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
1019 # get the node's dict |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
1020 d = self.db.getnode(self.classname, nodeid) |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
1021 |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
1022 # check for one of the special props |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1023 if propname == 'creation': |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1024 if 'creation' in d: |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
1025 return d['creation'] |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
1026 if not self.do_journal: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1027 raise ValueError('Journalling is disabled for this class') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1028 journal = self.db.getjournal(self.classname, nodeid) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1029 if journal: |
|
4342
94c992852f12
add in-memory hyperdb implementation to speed up testing
Richard Jones <richard@users.sourceforge.net>
parents:
4075
diff
changeset
|
1030 return journal[0][1] |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1031 else: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1032 # on the strange chance that there's no journal |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1033 return date.Date() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1034 if propname == 'activity': |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1035 if 'activity' in d: |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
1036 return d['activity'] |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
1037 if not self.do_journal: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1038 raise ValueError('Journalling is disabled for this class') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1039 journal = self.db.getjournal(self.classname, nodeid) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1040 if journal: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1041 return self.db.getjournal(self.classname, nodeid)[-1][1] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1042 else: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1043 # on the strange chance that there's no journal |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1044 return date.Date() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1045 if propname == 'creator': |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1046 if 'creator' in d: |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
1047 return d['creator'] |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
1048 if not self.do_journal: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1049 raise ValueError('Journalling is disabled for this class') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1050 journal = self.db.getjournal(self.classname, nodeid) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1051 if journal: |
| 5809 | 1052 num_re = re.compile(r'^\d+$') |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1053 value = journal[0][2] |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1054 if num_re.match(value): |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1055 return value |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1056 else: |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1057 # old-style "username" journal tag |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1058 try: |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1059 return self.db.user.lookup(value) |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1060 except KeyError: |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1061 # user's been retired, return admin |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1062 return '1' |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1063 else: |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1064 return self.db.getuid() |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1065 if propname == 'actor': |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1066 if 'actor' in d: |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1067 return d['actor'] |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1068 if not self.do_journal: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1069 raise ValueError('Journalling is disabled for this class') |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1070 journal = self.db.getjournal(self.classname, nodeid) |
|
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1071 if journal: |
| 5809 | 1072 num_re = re.compile(r'^\d+$') |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
1073 value = journal[-1][2] |
|
1176
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
1074 if num_re.match(value): |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
1075 return value |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
1076 else: |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
1077 # old-style "username" journal tag |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
1078 try: |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
1079 return self.db.user.lookup(value) |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
1080 except KeyError: |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
1081 # user's been retired, return admin |
|
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
1082 return '1' |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1083 else: |
|
1800
a3b1b1dcf639
Use getuid(), not figure_curuserid()
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
1794
diff
changeset
|
1084 return self.db.getuid() |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1085 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1086 # get the property (raises KeyErorr if invalid) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1087 prop = self.properties[propname] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1088 |
|
6179
a701c9c81597
Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6151
diff
changeset
|
1089 if isinstance(prop, hyperdb.Multilink) and prop.computed: |
|
a701c9c81597
Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6151
diff
changeset
|
1090 cls = self.db.getclass(prop.rev_classname) |
|
a701c9c81597
Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6151
diff
changeset
|
1091 ids = cls.find(**{prop.rev_propname: nodeid}) |
|
a701c9c81597
Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6151
diff
changeset
|
1092 return ids |
|
a701c9c81597
Fix rev_multilink properties search/retrieval
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6151
diff
changeset
|
1093 |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1094 if propname not in d: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1095 if default is _marker: |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1096 if isinstance(prop, hyperdb.Multilink): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1097 return [] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1098 else: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1099 return None |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1100 else: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1101 return default |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1102 |
|
1014
8816534e6a1a
Fixed nasty bug that was preventing changes to multilinks going through.
Richard Jones <richard@users.sourceforge.net>
parents:
1002
diff
changeset
|
1103 # return a dupe of the list so code doesn't get confused |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1104 if isinstance(prop, hyperdb.Multilink): |
|
6351
0db59cc2cd37
Enable testMultilinkOrdering check. Fix back_anydbm to pass.
John Rouillard <rouilj@ieee.org>
parents:
6238
diff
changeset
|
1105 ids = d[propname][:] |
|
0db59cc2cd37
Enable testMultilinkOrdering check. Fix back_anydbm to pass.
John Rouillard <rouilj@ieee.org>
parents:
6238
diff
changeset
|
1106 ids.sort(key=lambda x: int(x)) |
|
0db59cc2cd37
Enable testMultilinkOrdering check. Fix back_anydbm to pass.
John Rouillard <rouilj@ieee.org>
parents:
6238
diff
changeset
|
1107 return ids |
|
1014
8816534e6a1a
Fixed nasty bug that was preventing changes to multilinks going through.
Richard Jones <richard@users.sourceforge.net>
parents:
1002
diff
changeset
|
1108 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1109 return d[propname] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1110 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1111 def set(self, nodeid, **propvalues): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1112 """Modify a property on an existing node of this class. |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1113 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1114 'nodeid' must be the id of an existing node of this class or an |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1115 IndexError is raised. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1116 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1117 Each key in 'propvalues' must be the name of a property of this |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1118 class or a KeyError is raised. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1119 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1120 All values in 'propvalues' must be acceptable types for their |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1121 corresponding properties or a TypeError is raised. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1122 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1123 If the value of the key property is set, it must not collide with |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1124 other key strings or a ValueError is raised. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1125 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1126 If the value of a Link or Multilink property contains an invalid |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1127 node id, a ValueError is raised. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1128 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1129 These operations trigger detectors and can be vetoed. Attempts |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1130 to modify the "creation" or "activity" properties cause a KeyError. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1131 """ |
|
4347
0e33bf5571dc
make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents:
4342
diff
changeset
|
1132 if self.db.journaltag is None: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1133 raise hyperdb.DatabaseError(_('Database open read-only')) |
|
4347
0e33bf5571dc
make some more memorydb tests pass
Richard Jones <richard@users.sourceforge.net>
parents:
4342
diff
changeset
|
1134 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1135 self.fireAuditors('set', nodeid, propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1136 oldvalues = copy.deepcopy(self.db.getnode(self.classname, nodeid)) |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
1137 for name, prop in self.getprops(protected=0).items(): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1138 if name in oldvalues: |
|
2608
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
1139 continue |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1140 if isinstance(prop, hyperdb.Multilink): |
|
2608
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
1141 oldvalues[name] = [] |
|
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
1142 else: |
|
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
1143 oldvalues[name] = None |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1144 propvalues = self.set_inner(nodeid, **propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1145 self.fireReactors('set', nodeid, oldvalues) |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1146 return propvalues |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1147 |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1148 def set_inner(self, nodeid, **propvalues): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1149 """ Called by set, in-between the audit and react calls. |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1150 """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1151 if not propvalues: |
|
930
3c344e942055
Use same regex to split search terms as used to index text.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents:
927
diff
changeset
|
1152 return propvalues |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1153 |
|
5704
aa13a517cc63
Anydbm back end didn't list creator or actor as protected properties.
John Rouillard <rouilj@ieee.org>
parents:
5544
diff
changeset
|
1154 if ('creator' in propvalues or 'actor' in propvalues or |
|
aa13a517cc63
Anydbm back end didn't list creator or actor as protected properties.
John Rouillard <rouilj@ieee.org>
parents:
5544
diff
changeset
|
1155 'creation' in propvalues or 'activity' in propvalues): |
|
aa13a517cc63
Anydbm back end didn't list creator or actor as protected properties.
John Rouillard <rouilj@ieee.org>
parents:
5544
diff
changeset
|
1156 raise KeyError('"creator", "actor", "creation" and ' |
|
aa13a517cc63
Anydbm back end didn't list creator or actor as protected properties.
John Rouillard <rouilj@ieee.org>
parents:
5544
diff
changeset
|
1157 '"activity" are reserved') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1158 |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1159 if 'id' in propvalues: |
|
5378
35ea9b1efc14
Python 3 preparation: "raise" syntax.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5319
diff
changeset
|
1160 raise KeyError('"id" is reserved') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1161 |
|
6148
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1162 for p in propvalues: |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1163 prop = self.properties[p] |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1164 if prop.computed: |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1165 raise KeyError('"%s" is a computed property'%p) |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1166 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1167 if self.db.journaltag is None: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1168 raise hyperdb.DatabaseError(_('Database open read-only')) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1169 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1170 node = self.db.getnode(self.classname, nodeid) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1171 if self.db.RETIRED_FLAG in node: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1172 raise IndexError |
| 5809 | 1173 num_re = re.compile(r'^\d+$') |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1174 |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1175 # if the journal value is to be different, store it in here |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1176 journalvalues = {} |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1177 |
|
5112
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1178 # omit quiet properties from history/changelog |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1179 quiet_props = [] |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1180 |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1181 # list() propvalues 'cos it might be modified by the loop |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1182 for propname, value in list(propvalues.items()): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1183 # check to make sure we're not duplicating an existing key |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1184 if propname == self.key and node[propname] != value: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1185 try: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1186 self.lookup(value) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1187 except KeyError: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1188 pass |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1189 else: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1190 raise ValueError('node with key "%s" exists'%value) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1191 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1192 # this will raise the KeyError if the property isn't valid |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1193 # ... we don't use getprops() here because we only care about |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1194 # the writeable properties. |
|
1174
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
1195 try: |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
1196 prop = self.properties[propname] |
|
8e318dfaf479
Verify contents of tracker module when the tracker is opened
Richard Jones <richard@users.sourceforge.net>
parents:
1170
diff
changeset
|
1197 except KeyError: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1198 raise KeyError('"%s" has no property named "%s"'%( |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1199 self.classname, propname)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1200 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1201 # if the value's the same as the existing value, no sense in |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1202 # doing anything |
|
1304
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1303
diff
changeset
|
1203 current = node.get(propname, None) |
|
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1303
diff
changeset
|
1204 if value == current: |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1205 del propvalues[propname] |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1206 continue |
|
1304
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1303
diff
changeset
|
1207 journalvalues[propname] = current |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1208 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1209 # do stuff based on the prop type |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1210 if isinstance(prop, hyperdb.Link): |
|
927
51519406b73e
web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents:
924
diff
changeset
|
1211 link_class = prop.classname |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1212 # if it isn't a number, it's a key |
|
927
51519406b73e
web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents:
924
diff
changeset
|
1213 if value is not None and not isinstance(value, type('')): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1214 raise ValueError('property "%s" link value be a string'%( |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1215 propname)) |
|
927
51519406b73e
web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents:
924
diff
changeset
|
1216 if isinstance(value, type('')) and not num_re.match(value): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1217 try: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1218 value = self.db.classes[link_class].lookup(value) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1219 except (TypeError, KeyError): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1220 raise IndexError('new property "%s": %s not a %s'%( |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1221 propname, value, prop.classname)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1222 |
|
927
51519406b73e
web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents:
924
diff
changeset
|
1223 if (value is not None and |
|
51519406b73e
web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents:
924
diff
changeset
|
1224 not self.db.getclass(link_class).hasnode(value)): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1225 raise IndexError('%s has no node %s'%(link_class, |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1226 value)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1227 |
|
927
51519406b73e
web forms may now unset Link values (like assignedto)
Richard Jones <richard@users.sourceforge.net>
parents:
924
diff
changeset
|
1228 if self.do_journal and prop.do_journal: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1229 # register the unlink with the old linked node |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1230 if propname in node and node[propname] is not None: |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1231 self.db.addjournal(link_class, node[propname], 'unlink', |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1232 (self.classname, nodeid, propname)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1233 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1234 # register the link with the newly linked node |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1235 if value is not None: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1236 self.db.addjournal(link_class, value, 'link', |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1237 (self.classname, nodeid, propname)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1238 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1239 elif isinstance(prop, hyperdb.Multilink): |
|
3872
34128a809e22
Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3869
diff
changeset
|
1240 if value is None: |
|
34128a809e22
Allow multilinks to take None (treated as an empty list).
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3869
diff
changeset
|
1241 value = [] |
|
5467
630a00b20394
strings have an __iter__ method on Python 3, but we don't want to consider them iterable
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5444
diff
changeset
|
1242 if not hasattr(value, '__iter__') or type(value) == type(''): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1243 raise TypeError('new property "%s" not an iterable of' |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1244 ' ids'%propname) |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1245 link_class = self.properties[propname].classname |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1246 l = [] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1247 for entry in value: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1248 # if it isn't a number, it's a key |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1249 if type(entry) != type(''): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1250 raise ValueError('new property "%s" link value ' |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1251 'must be a string'%propname) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1252 if not num_re.match(entry): |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1253 try: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1254 entry = self.db.classes[link_class].lookup(entry) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1255 except (TypeError, KeyError): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1256 raise IndexError('new property "%s": %s not a %s'%( |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1257 propname, entry, |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1258 self.properties[propname].classname)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1259 l.append(entry) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1260 value = l |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1261 propvalues[propname] = value |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1262 |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1263 # figure the journal entry for this property |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1264 add = [] |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1265 remove = [] |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1266 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1267 # handle removals |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1268 if propname in node: |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1269 l = node[propname] |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1270 else: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1271 l = [] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1272 for id in l[:]: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1273 if id in value: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1274 continue |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1275 # register the unlink with the old linked node |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1276 if self.do_journal and self.properties[propname].do_journal: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1277 self.db.addjournal(link_class, id, 'unlink', |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1278 (self.classname, nodeid, propname)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1279 l.remove(id) |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1280 remove.append(id) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1281 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1282 # handle additions |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1283 for id in value: |
|
902
b0d3d3535998
Bugger it. Here's the current shape of the new security implementation.
Richard Jones <richard@users.sourceforge.net>
parents:
891
diff
changeset
|
1284 if not self.db.getclass(link_class).hasnode(id): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1285 raise IndexError('%s has no node %s'%(link_class, |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1286 id)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1287 if id in l: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1288 continue |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1289 # register the link with the newly linked node |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1290 if self.do_journal and self.properties[propname].do_journal: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1291 self.db.addjournal(link_class, id, 'link', |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1292 (self.classname, nodeid, propname)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1293 l.append(id) |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1294 add.append(id) |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1295 |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1296 # figure the journal entry |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1297 l = [] |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1298 if add: |
|
967
dd35bab19dd9
use more robust date stamp comparisons in pack(), make journal smaller too
Richard Jones <richard@users.sourceforge.net>
parents:
964
diff
changeset
|
1299 l.append(('+', add)) |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1300 if remove: |
|
967
dd35bab19dd9
use more robust date stamp comparisons in pack(), make journal smaller too
Richard Jones <richard@users.sourceforge.net>
parents:
964
diff
changeset
|
1301 l.append(('-', remove)) |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1302 if l: |
|
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1303 journalvalues[propname] = tuple(l) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1304 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1305 elif isinstance(prop, hyperdb.String): |
|
1383
f19dde90e473
applied unicode patch
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1351
diff
changeset
|
1306 if value is not None and type(value) != type('') and type(value) != type(u''): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1307 raise TypeError('new property "%s" not a ' |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1308 'string'%propname) |
|
2892
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2736
diff
changeset
|
1309 if prop.indexme: |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2736
diff
changeset
|
1310 self.db.indexer.add_text((self.classname, nodeid, propname), |
|
2eae5848912d
always honor indexme property on Strings (patch [SF#063711])
Richard Jones <richard@users.sourceforge.net>
parents:
2736
diff
changeset
|
1311 value) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1312 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1313 elif isinstance(prop, hyperdb.Password): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1314 if not isinstance(value, password.Password): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1315 raise TypeError('new property "%s" not a ' |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1316 'Password'%propname) |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1317 propvalues[propname] = value |
|
4483
22bc0426e348
Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4480
diff
changeset
|
1318 journalvalues[propname] = \ |
|
22bc0426e348
Second patch from issue2550688 -- with some changes:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4480
diff
changeset
|
1319 current and password.JournalPassword(current) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1320 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1321 elif value is not None and isinstance(prop, hyperdb.Date): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1322 if not isinstance(value, date.Date): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1323 raise TypeError('new property "%s" not a ' |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1324 'Date'%propname) |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1325 propvalues[propname] = value |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1326 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1327 elif value is not None and isinstance(prop, hyperdb.Interval): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1328 if not isinstance(value, date.Interval): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1329 raise TypeError('new property "%s" not an ' |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1330 'Interval'%propname) |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1331 propvalues[propname] = value |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1332 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1333 elif value is not None and isinstance(prop, hyperdb.Number): |
|
880
de3da99a7c02
Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents:
869
diff
changeset
|
1334 try: |
|
890
a568596dbea7
Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
887
diff
changeset
|
1335 float(value) |
|
887
e7169d6e6e45
added tests for number type too
Richard Jones <richard@users.sourceforge.net>
parents:
886
diff
changeset
|
1336 except ValueError: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1337 raise TypeError('new property "%s" not ' |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1338 'numeric'%propname) |
|
880
de3da99a7c02
Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents:
869
diff
changeset
|
1339 |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1340 elif value is not None and isinstance(prop, hyperdb.Integer): |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1341 try: |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1342 int(value) |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1343 except ValueError: |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1344 raise TypeError('new property "%s" not ' |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1345 'numeric'%propname) |
|
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1346 |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1347 elif value is not None and isinstance(prop, hyperdb.Boolean): |
|
890
a568596dbea7
Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
887
diff
changeset
|
1348 try: |
|
a568596dbea7
Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
887
diff
changeset
|
1349 int(value) |
|
a568596dbea7
Unit tests and a few fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
887
diff
changeset
|
1350 except ValueError: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1351 raise TypeError('new property "%s" not ' |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1352 'boolean'%propname) |
|
880
de3da99a7c02
Add Number and Boolean types to hyperdb.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents:
869
diff
changeset
|
1353 |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1354 node[propname] = value |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1355 |
|
5112
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1356 # record quiet properties to omit from history/changelog |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1357 if prop.quiet: |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1358 quiet_props.append(propname) |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1359 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1360 # nothing to do? |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1361 if not propvalues: |
|
930
3c344e942055
Use same regex to split search terms as used to index text.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents:
927
diff
changeset
|
1362 return propvalues |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1363 |
|
3083
b9a819ef7554
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3052
diff
changeset
|
1364 # update the activity time |
|
b9a819ef7554
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3052
diff
changeset
|
1365 node['activity'] = date.Date() |
|
b9a819ef7554
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3052
diff
changeset
|
1366 node['actor'] = self.db.getuid() |
|
b9a819ef7554
actor/activity update moved from Database.setnode() to Class.set_inner()
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3052
diff
changeset
|
1367 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1368 # do the set, and journal it |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1369 self.db.setnode(self.classname, nodeid, node) |
|
869
6d98bec4e52e
fixed the journal bloat from multilink changes
Richard Jones <richard@users.sourceforge.net>
parents:
862
diff
changeset
|
1370 |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
1371 if self.do_journal: |
|
1304
61ad556cfc8d
working toward 0.5.2 release
Richard Jones <richard@users.sourceforge.net>
parents:
1303
diff
changeset
|
1372 self.db.addjournal(self.classname, nodeid, 'set', journalvalues) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1373 |
|
5112
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1374 # remove quiet properties from output |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1375 for propname in quiet_props: |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1376 if propname in propvalues: |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1377 del propvalues[propname] |
|
8901cc4ef0e0
- issue1714899: Feature Request: Optional Change Note. Added a new
John Rouillard <rouilj@ieee.org>
parents:
5107
diff
changeset
|
1378 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
1379 return propvalues |
|
930
3c344e942055
Use same regex to split search terms as used to index text.
Gordon B. McMillan <gmcm@users.sourceforge.net>
parents:
927
diff
changeset
|
1380 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1381 def retire(self, nodeid): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1382 """Retire a node. |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1383 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1384 The properties on the node remain available from the get() method, |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1385 and the node's id is never reused. |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1386 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1387 Retired nodes are not returned by the find(), list(), or lookup() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1388 methods, and other nodes may reuse the values of their key properties. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1389 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1390 These operations trigger detectors and can be vetoed. Attempts |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1391 to modify the "creation" or "activity" properties cause a KeyError. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1392 """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1393 if self.db.journaltag is None: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1394 raise hyperdb.DatabaseError(_('Database open read-only')) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1395 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1396 self.fireAuditors('retire', nodeid, None) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1397 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1398 node = self.db.getnode(self.classname, nodeid) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1399 node[self.db.RETIRED_FLAG] = 1 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1400 self.db.setnode(self.classname, nodeid, node) |
|
860
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
1401 if self.do_journal: |
|
2df32a73eb45
Implemented a switch to disable journalling for a Class.
Richard Jones <richard@users.sourceforge.net>
parents:
858
diff
changeset
|
1402 self.db.addjournal(self.classname, nodeid, 'retired', None) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1403 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1404 self.fireReactors('retire', nodeid, None) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1405 |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1406 def restore(self, nodeid): |
|
5107
33ac07a7ef96
spelling error fixed in comment
John Rouillard <rouilj@ieee.org>
parents:
5100
diff
changeset
|
1407 """Restore a retired node. |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1408 |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1409 Make node available for all operations like it was before retirement. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1410 """ |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1411 if self.db.journaltag is None: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1412 raise hyperdb.DatabaseError(_('Database open read-only')) |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1413 |
|
1523
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1414 node = self.db.getnode(self.classname, nodeid) |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1415 # check if key property was overrided |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1416 key = self.getkey() |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1417 try: |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1418 id = self.lookup(node[key]) |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1419 except KeyError: |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1420 pass |
|
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1421 else: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1422 raise KeyError("Key property (%s) of retired node clashes " |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1423 "with existing one (%s)" % (key, node[key])) |
|
1523
63aa7be52d2c
checked to make sure that the restored item doesn't clash...
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1519
diff
changeset
|
1424 # Now we can safely restore node |
|
1519
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1425 self.fireAuditors('restore', nodeid, None) |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1426 del node[self.db.RETIRED_FLAG] |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1427 self.db.setnode(self.classname, nodeid, node) |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1428 if self.do_journal: |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1429 self.db.addjournal(self.classname, nodeid, 'restored', None) |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1430 |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1431 self.fireReactors('restore', nodeid, None) |
|
6fede2aa6a12
added ability to restore retired nodes
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1505
diff
changeset
|
1432 |
|
1476
5a01e90b7dc9
fixed export/import of retired nodes [SF#685273]
Richard Jones <richard@users.sourceforge.net>
parents:
1467
diff
changeset
|
1433 def is_retired(self, nodeid, cldb=None): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1434 """Return true if the node is retired. |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1435 """ |
|
1476
5a01e90b7dc9
fixed export/import of retired nodes [SF#685273]
Richard Jones <richard@users.sourceforge.net>
parents:
1467
diff
changeset
|
1436 node = self.db.getnode(self.classname, nodeid, cldb) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1437 if self.db.RETIRED_FLAG in node: |
|
940
301a02ea6020
added is_retired query to Class
Richard Jones <richard@users.sourceforge.net>
parents:
930
diff
changeset
|
1438 return 1 |
|
301a02ea6020
added is_retired query to Class
Richard Jones <richard@users.sourceforge.net>
parents:
930
diff
changeset
|
1439 return 0 |
|
301a02ea6020
added is_retired query to Class
Richard Jones <richard@users.sourceforge.net>
parents:
930
diff
changeset
|
1440 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1441 def destroy(self, nodeid): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1442 """Destroy a node. |
|
1333
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
1443 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1444 WARNING: this method should never be used except in extremely rare |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1445 situations where there could never be links to the node being |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1446 deleted |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1447 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1448 WARNING: use retire() instead |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1449 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1450 WARNING: the properties of this node will not be available ever again |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1451 |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1452 WARNING: really, use retire() instead |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1453 |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1454 Well, I think that's enough warnings. This method exists mostly to |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1455 support the session storage of the cgi interface. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1456 """ |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1457 if self.db.journaltag is None: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1458 raise hyperdb.DatabaseError(_('Database open read-only')) |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1459 self.db.destroynode(self.classname, nodeid) |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
1460 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1461 # Locating nodes: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1462 def hasnode(self, nodeid): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1463 """Determine if the given nodeid actually exists |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1464 """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1465 return self.db.hasnode(self.classname, nodeid) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1466 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1467 def setkey(self, propname): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1468 """Select a String property of this class to be the key property. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1469 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1470 'propname' must be the name of a String property of this class or |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1471 None, or a TypeError is raised. The values of the key property on |
|
862
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
1472 all existing nodes must be unique or a ValueError is raised. If the |
|
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
1473 property doesn't exist, KeyError is raised. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1474 """ |
|
862
37fb48c3a136
Did some old TODOs
Richard Jones <richard@users.sourceforge.net>
parents:
860
diff
changeset
|
1475 prop = self.getprops()[propname] |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1476 if not isinstance(prop, hyperdb.String): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1477 raise TypeError('key properties must be String') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1478 self.key = propname |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1479 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1480 def getkey(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1481 """Return the name of the key property for this class or None.""" |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1482 return self.key |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1483 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1484 # TODO: set up a separate index db file for this? profile? |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1485 def lookup(self, keyvalue): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1486 """Locate a particular node by its key property and return its id. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1487 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1488 If this class has no key property, a TypeError is raised. If the |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1489 'keyvalue' matches one of the values for the key property among |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1490 the nodes in this class, the matching node's id is returned; |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1491 otherwise a KeyError is raised. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1492 """ |
|
969
ddcb527491ba
Consistent quoting
Richard Jones <richard@users.sourceforge.net>
parents:
967
diff
changeset
|
1493 if not self.key: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1494 raise TypeError('No key property set for ' |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1495 'class %s'%self.classname) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1496 cldb = self.db.getclassdb(self.classname) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1497 try: |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1498 for nodeid in self.getnodeids(cldb): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1499 node = self.db.getnode(self.classname, nodeid, cldb) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1500 if self.db.RETIRED_FLAG in node: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1501 continue |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1502 if self.key not in node: |
|
2606
17eb5aeada7f
fix bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2601
diff
changeset
|
1503 continue |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1504 if node[self.key] == keyvalue: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1505 return nodeid |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1506 finally: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1507 cldb.close() |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1508 raise KeyError('No key (%s) value "%s" for "%s"'%(self.key, |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1509 keyvalue, self.classname)) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1510 |
|
1103
db787cef1385
handled some XXXs
Richard Jones <richard@users.sourceforge.net>
parents:
1099
diff
changeset
|
1511 # change from spec - allows multiple props to match |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1512 def find(self, **propspec): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1513 """Get the ids of nodes in this class which link to the given nodes. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1514 |
|
3239
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3166
diff
changeset
|
1515 'propspec' consists of keyword args propname=nodeid or |
|
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3166
diff
changeset
|
1516 propname={nodeid:1, } |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1195
diff
changeset
|
1517 'propname' must be the name of a property in this class, or a |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1195
diff
changeset
|
1518 KeyError is raised. That property must be a Link or |
|
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1195
diff
changeset
|
1519 Multilink property, or a TypeError is raised. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1520 |
|
3239
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3166
diff
changeset
|
1521 Any node in this class whose 'propname' property links to any of |
|
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3166
diff
changeset
|
1522 the nodeids will be returned. Examples:: |
|
1222
bc3bc3248dd1
added Class.find() unit test, fixed implementations
Richard Jones <richard@users.sourceforge.net>
parents:
1195
diff
changeset
|
1523 |
|
3239
440f0a6a2e3c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3166
diff
changeset
|
1524 db.issue.find(messages='1') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1525 db.issue.find(messages={'1':1,'3':1}, files={'7':1}) |
|
6148
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1526 db.issue.find(messages=('1','3'), files=('7',)) |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1527 db.issue.find(messages=['1','3'], files=['7']) |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1528 """ |
|
6427
f08907bfd5a1
Fix find() with anydbm. Add fast return shortcut.
John Rouillard <rouilj@ieee.org>
parents:
6412
diff
changeset
|
1529 # shortcut |
|
f08907bfd5a1
Fix find() with anydbm. Add fast return shortcut.
John Rouillard <rouilj@ieee.org>
parents:
6412
diff
changeset
|
1530 if not propspec: |
|
f08907bfd5a1
Fix find() with anydbm. Add fast return shortcut.
John Rouillard <rouilj@ieee.org>
parents:
6412
diff
changeset
|
1531 return [] |
|
f08907bfd5a1
Fix find() with anydbm. Add fast return shortcut.
John Rouillard <rouilj@ieee.org>
parents:
6412
diff
changeset
|
1532 |
|
f08907bfd5a1
Fix find() with anydbm. Add fast return shortcut.
John Rouillard <rouilj@ieee.org>
parents:
6412
diff
changeset
|
1533 # validate the args |
|
f08907bfd5a1
Fix find() with anydbm. Add fast return shortcut.
John Rouillard <rouilj@ieee.org>
parents:
6412
diff
changeset
|
1534 props = self.getprops() |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
1535 for propname, itemids in propspec.items(): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1536 # check the prop is OK |
|
6427
f08907bfd5a1
Fix find() with anydbm. Add fast return shortcut.
John Rouillard <rouilj@ieee.org>
parents:
6412
diff
changeset
|
1537 prop = props[propname] |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1538 if not isinstance(prop, hyperdb.Link) and not isinstance(prop, hyperdb.Multilink): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1539 raise TypeError("'%s' not a Link/Multilink " |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1540 "property"%propname) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1541 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1542 # ok, now do the find |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1543 cldb = self.db.getclassdb(self.classname) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1544 l = [] |
|
6151
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1545 rev_multilinks = [] |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1546 try: |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1547 for id in self.getnodeids(db=cldb): |
|
1563
e2a8ce4d2317
Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents:
1558
diff
changeset
|
1548 item = self.db.getnode(self.classname, id, db=cldb) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1549 if self.db.RETIRED_FLAG in item: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1550 continue |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
1551 for propname, itemids in propspec.items(): |
|
1563
e2a8ce4d2317
Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents:
1558
diff
changeset
|
1552 if type(itemids) is not type({}): |
|
6148
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1553 if itemids is None or isinstance(itemids, type("")): |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1554 itemids = {itemids:1} |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1555 else: |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1556 itemids = dict.fromkeys(itemids) |
|
1563
e2a8ce4d2317
Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents:
1558
diff
changeset
|
1557 |
|
2450
c45ed2413044
fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents:
2437
diff
changeset
|
1558 # special case if the item doesn't have this property |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1559 if propname not in item: |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1560 if None in itemids: |
|
2450
c45ed2413044
fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents:
2437
diff
changeset
|
1561 l.append(id) |
|
c45ed2413044
fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents:
2437
diff
changeset
|
1562 break |
|
c45ed2413044
fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents:
2437
diff
changeset
|
1563 continue |
|
c45ed2413044
fixed lookup of "missing" Link values for new props in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents:
2437
diff
changeset
|
1564 |
|
1563
e2a8ce4d2317
Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents:
1558
diff
changeset
|
1565 # grab the property definition and its value on this item |
|
6427
f08907bfd5a1
Fix find() with anydbm. Add fast return shortcut.
John Rouillard <rouilj@ieee.org>
parents:
6412
diff
changeset
|
1566 prop = props[propname] |
|
1563
e2a8ce4d2317
Class.find() may now find unset Links [SF#700620]
Richard Jones <richard@users.sourceforge.net>
parents:
1558
diff
changeset
|
1567 value = item[propname] |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1568 if isinstance(prop, hyperdb.Link) and value in itemids: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1569 l.append(id) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1570 break |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1571 elif isinstance(prop, hyperdb.Multilink): |
|
6151
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1572 if prop.rev_property: |
|
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1573 rev_multilinks.append ((prop, itemids)) |
|
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1574 continue |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1575 hit = 0 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1576 for v in value: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1577 if v in itemids: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1578 l.append(id) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1579 hit = 1 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1580 break |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1581 if hit: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1582 break |
|
6151
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1583 for prop, itemids in rev_multilinks: |
|
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1584 rprop = prop.rev_property |
|
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1585 fun = l.append |
|
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1586 if isinstance (rprop, hyperdb.Multilink): |
|
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1587 fun = l.extend |
|
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1588 for id in itemids: |
|
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1589 fun(rprop.cls.get(id, rprop.name)) |
|
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1590 if rev_multilinks: |
|
ff059afae50a
Make 'find' work for rev_multilink properties
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6148
diff
changeset
|
1591 l = list(sorted(set(l))) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1592 finally: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1593 cldb.close() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1594 return l |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1595 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1596 def stringFind(self, **requirements): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1597 """Locate a particular node by matching a set of its String |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1598 properties in a caseless search. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1599 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1600 If the property is not a String property, a TypeError is raised. |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1601 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1602 The return is a list of the id of all nodes that match. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1603 """ |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1604 for propname in requirements: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1605 prop = self.properties[propname] |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
1606 if not isinstance(prop, hyperdb.String): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1607 raise TypeError("'%s' not a String property"%propname) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1608 requirements[propname] = requirements[propname].lower() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1609 l = [] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1610 cldb = self.db.getclassdb(self.classname) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1611 try: |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1612 for nodeid in self.getnodeids(cldb): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1613 node = self.db.getnode(self.classname, nodeid, cldb) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1614 if self.db.RETIRED_FLAG in node: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1615 continue |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
1616 for key, value in requirements.items(): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1617 if key not in node: |
|
1296
34b1e6490e65
hardening for stringFind with missing props
Richard Jones <richard@users.sourceforge.net>
parents:
1252
diff
changeset
|
1618 break |
|
905
502a5ae11cc5
Very close now. The cgi and mailgw now use the new security API.
Richard Jones <richard@users.sourceforge.net>
parents:
902
diff
changeset
|
1619 if node[key] is None or node[key].lower() != value: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1620 break |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1621 else: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1622 l.append(nodeid) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1623 finally: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1624 cldb.close() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1625 return l |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1626 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1627 def list(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1628 """ Return a list of the ids of the active nodes in this class. |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1629 """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1630 l = [] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1631 cn = self.classname |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1632 cldb = self.db.getclassdb(cn) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1633 try: |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1634 for nodeid in self.getnodeids(cldb): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1635 node = self.db.getnode(cn, nodeid, cldb) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1636 if self.db.RETIRED_FLAG in node: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1637 continue |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1638 l.append(nodeid) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1639 finally: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1640 cldb.close() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1641 l.sort() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1642 return l |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1643 |
|
3486
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1644 def getnodeids(self, db=None, retired=None): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1645 """ Return a list of ALL nodeids |
|
3486
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1646 |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1647 Set retired=None to get all nodes. Otherwise it'll get all the |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1648 retired or non-retired nodes, depending on the flag. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1649 """ |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1650 res = [] |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1651 |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1652 # start off with the new nodes |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1653 if self.classname in self.db.newnodes: |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1654 res.extend(self.db.newnodes[self.classname]) |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1655 |
|
3486
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1656 must_close = False |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1657 if db is None: |
|
1486
f5f60c75a458
added test for error in sqlite backend, and fixed *dbm backend error
Richard Jones <richard@users.sourceforge.net>
parents:
1484
diff
changeset
|
1658 db = self.db.getclassdb(self.classname) |
|
3906
666b70676ec6
destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3897
diff
changeset
|
1659 must_close = True |
|
3486
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1660 try: |
|
5444
167f0d25ea8e
Python 3 preparation: convert dbm keys back from bytes to strings.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5395
diff
changeset
|
1661 res.extend(map(b2s, db.keys())) |
|
3486
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1662 |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1663 # remove the uncommitted, destroyed nodes |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1664 if self.classname in self.db.destroyednodes: |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1665 for nodeid in self.db.destroyednodes[self.classname]: |
|
5011
d5da643b3d25
Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents:
4995
diff
changeset
|
1666 if nodeid in db: |
|
3486
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1667 res.remove(nodeid) |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1668 |
|
3486
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1669 # check retired flag |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1670 if retired is False or retired is True: |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1671 l = [] |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1672 for nodeid in res: |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1673 node = self.db.getnode(self.classname, nodeid, db) |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1674 is_ret = self.db.RETIRED_FLAG in node |
|
3486
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1675 if retired == is_ret: |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1676 l.append(nodeid) |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1677 res = l |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1678 finally: |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1679 if must_close: |
|
34ada15b9936
all backends implement the retired check in getnodeids [SF#1290560]
Richard Jones <richard@users.sourceforge.net>
parents:
3476
diff
changeset
|
1680 db.close() |
|
1484
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1681 return res |
|
b3f2484babce
fixes to import/export
Richard Jones <richard@users.sourceforge.net>
parents:
1476
diff
changeset
|
1682 |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1683 def _filter(self, search_matches, filterspec, proptree, |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1684 num_re = re.compile(r'^\d+$'), retired=False, |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1685 exact_match_spec={}): |
|
5318
506c7ee9a385
Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5248
diff
changeset
|
1686 """Return a list of the ids of the nodes in this class that |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1687 match the 'filter' spec, sorted by the group spec and then the |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1688 sort spec. |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1689 |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1690 "filterspec" is {propname: value(s)} |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1691 same for "exact_match_spec". The latter specifies exact matching |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1692 for String type while String types in "filterspec" are searched |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1693 for as case insensitive substring match. |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1694 |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1695 "sort" and "group" are (dir, prop) where dir is '+', '-' or None |
|
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1696 and prop is a prop name or None |
| 924 | 1697 |
| 4044 | 1698 "search_matches" is a sequence type or None |
|
1170
af104fa52746
Added some words to the installation doc about choosing backends.
Richard Jones <richard@users.sourceforge.net>
parents:
1159
diff
changeset
|
1699 |
|
5318
506c7ee9a385
Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5248
diff
changeset
|
1700 "retired" specifies if we should find only non-retired nodes |
|
506c7ee9a385
Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5248
diff
changeset
|
1701 (default) or only retired node (value True) or all nodes. |
|
506c7ee9a385
Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5248
diff
changeset
|
1702 |
|
3455
e01bc6d65fa9
fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents:
3419
diff
changeset
|
1703 The filter must match all properties specificed. If the property |
|
e01bc6d65fa9
fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents:
3419
diff
changeset
|
1704 value to match is a list: |
|
3906
666b70676ec6
destroy blobfiles if they exist
Justus Pendleton <jpend@users.sourceforge.net>
parents:
3897
diff
changeset
|
1705 |
|
3455
e01bc6d65fa9
fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents:
3419
diff
changeset
|
1706 1. String properties must match all elements in the list, and |
|
e01bc6d65fa9
fixed documentation of filter()...
Richard Jones <richard@users.sourceforge.net>
parents:
3419
diff
changeset
|
1707 2. Other properties must match any of the elements in the list. |
|
2005
fc52d57c6c3e
documentation cleanup
Richard Jones <richard@users.sourceforge.net>
parents:
1955
diff
changeset
|
1708 """ |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
1709 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
1710 start_t = time.time() |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
1711 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1712 cn = self.classname |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1713 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1714 # optimise filterspec |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1715 l = [] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1716 props = self.getprops() |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1717 LINK = 'spec:link' |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1718 MULTILINK = 'spec:multilink' |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1719 STRING = 'spec:string' |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1720 DATE = 'spec:date' |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1721 INTERVAL = 'spec:interval' |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1722 OTHER = 'spec:other' |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1723 |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1724 for exact, filtertype in enumerate((filterspec, exact_match_spec)): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1725 for k, v in filtertype.items(): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1726 propclass = props[k] |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1727 if isinstance(propclass, hyperdb.Link): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1728 if type(v) is not type([]): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1729 v = [v] |
|
6412
a0c0ee3ed8b1
Tests for Link expressions
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6401
diff
changeset
|
1730 l.append((LINK, k, v)) |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1731 elif isinstance(propclass, hyperdb.Multilink): |
|
6148
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1732 # If it's a reverse multilink, we've already |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1733 # computed the ids of our own class. |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1734 if propclass.rev_property: |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1735 l.append((OTHER, 'id', v)) |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1736 else: |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1737 # the value -1 is a special "not set" sentinel |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1738 if v in ('-1', ['-1']): |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1739 v = [] |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1740 elif type(v) is not type([]): |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1741 v = [v] |
|
8497bf3f23a1
Allow to define reverse Multilinks
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6118
diff
changeset
|
1742 l.append((MULTILINK, k, v)) |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1743 elif isinstance(propclass, hyperdb.String) and k != 'id': |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1744 if type(v) is not type([]): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1745 v = [v] |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1746 for x in v: |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1747 if exact: |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1748 l.append((STRING, k, x)) |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1749 else: |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1750 # simple glob searching |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1751 x = re.sub(r'([\|\{\}\\\.\+\[\]\(\)])', r'\\\1', x) |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1752 x = x.replace('?', '.') |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1753 x = x.replace('*', '.*?') |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1754 l.append((STRING, k, re.compile(x, re.I))) |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1755 elif isinstance(propclass, hyperdb.Date): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1756 try: |
| 6118 | 1757 ranges = [] |
| 1758 for d in v.split(','): | |
| 1759 if d == '-': | |
| 1760 ranges.append(None) | |
| 1761 continue | |
| 1762 date_rng = propclass.range_from_raw(d, self.db) | |
| 1763 ranges.append(date_rng) | |
| 1764 l.append((DATE, k, ranges)) | |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1765 except ValueError: |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1766 # If range creation fails - ignore that search parameter |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1767 pass |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1768 elif isinstance(propclass, hyperdb.Interval): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1769 try: |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1770 intv_rng = date.Range(v, date.Interval) |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1771 l.append((INTERVAL, k, intv_rng)) |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1772 except ValueError: |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1773 # If range creation fails - ignore that search parameter |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1774 pass |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
1775 |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1776 elif isinstance(propclass, hyperdb.Boolean): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1777 if type(v) == type(""): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1778 v = v.split(',') |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1779 if type(v) != type([]): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1780 v = [v] |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1781 bv = [] |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1782 for val in v: |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1783 if type(val) is type(''): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1784 bv.append(propclass.from_raw (val)) |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1785 else: |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1786 bv.append(val) |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1787 l.append((OTHER, k, bv)) |
|
2453
0e2a0c2c8142
allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents:
2450
diff
changeset
|
1788 |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1789 elif k == 'id': |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1790 if type(v) != type([]): |
|
4364
0e81742d0e2f
- unify number searching across backends
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4357
diff
changeset
|
1791 v = v.split(',') |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1792 l.append((OTHER, k, [str(int(val)) for val in v])) |
|
2453
0e2a0c2c8142
allow list of values for id, Number and Boolean filtering in anydbm backend
Richard Jones <richard@users.sourceforge.net>
parents:
2450
diff
changeset
|
1793 |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1794 elif isinstance(propclass, hyperdb.Number): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1795 if type(v) != type([]): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1796 try : |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1797 v = v.split(',') |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1798 except AttributeError : |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1799 v = [v] |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1800 l.append((OTHER, k, [float(val) for val in v])) |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1801 |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1802 elif isinstance(propclass, hyperdb.Integer): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1803 if type(v) != type([]): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1804 try : |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1805 v = v.split(',') |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1806 except AttributeError : |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1807 v = [v] |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1808 l.append((OTHER, k, [int(val) for val in v])) |
|
5067
e424987d294a
Add support for an integer type to join the existing number type.
John Rouillard <rouilj@ieee.org>
parents:
5041
diff
changeset
|
1809 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1810 filterspec = l |
|
4480
1613754d2646
Fix first part of Password handling security issue2550688
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
1811 |
|
5318
506c7ee9a385
Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5248
diff
changeset
|
1812 # now, find all the nodes that pass filtering |
|
2239
c8a06e10e2c6
much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
1813 matches = [] |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1814 cldb = self.db.getclassdb(cn) |
|
2252
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1815 t = 0 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1816 try: |
| 924 | 1817 # TODO: only full-scan once (use items()) |
|
5318
506c7ee9a385
Add a 'retired' parameter to Class.filter
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5248
diff
changeset
|
1818 for nodeid in self.getnodeids(cldb, retired=retired): |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1819 node = self.db.getnode(cn, nodeid, cldb) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1820 # apply filter |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1821 for t, k, v in filterspec: |
|
1303
71be6588904f
fixed filtering by id in anydbm
Richard Jones <richard@users.sourceforge.net>
parents:
1296
diff
changeset
|
1822 # handle the id prop |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1823 if k == 'id': |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1824 if nodeid not in v: |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1825 break |
|
1303
71be6588904f
fixed filtering by id in anydbm
Richard Jones <richard@users.sourceforge.net>
parents:
1296
diff
changeset
|
1826 continue |
|
71be6588904f
fixed filtering by id in anydbm
Richard Jones <richard@users.sourceforge.net>
parents:
1296
diff
changeset
|
1827 |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1828 # get the node value |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1829 nv = node.get(k, None) |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1830 |
|
2617
33fffbf7ae68
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2608
diff
changeset
|
1831 match = 0 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1832 |
| 924 | 1833 # now apply the property filter |
| 1834 if t == LINK: | |
| 1835 # link - if this node's property doesn't appear in the | |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1836 # filterspec's nodeid list, skip it |
|
6412
a0c0ee3ed8b1
Tests for Link expressions
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6401
diff
changeset
|
1837 expr = Expression(v, is_link=True) |
|
a0c0ee3ed8b1
Tests for Link expressions
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6401
diff
changeset
|
1838 if expr.evaluate(nv): |
|
a0c0ee3ed8b1
Tests for Link expressions
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6401
diff
changeset
|
1839 match = 1 |
| 924 | 1840 elif t == MULTILINK: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1841 # multilink - if any of the nodeids required by the |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1842 # filterspec aren't in this node's property, then skip |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1843 # it |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1844 nv = node.get(k, []) |
|
1555
948c7764d46c
implemented ability to search for multilink properties with no value (not in mk)
Richard Jones <richard@users.sourceforge.net>
parents:
1523
diff
changeset
|
1845 |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1846 # check for matching the absence of multilink values |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1847 if not v: |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1848 match = not nv |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1849 else: |
|
4466
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4447
diff
changeset
|
1850 # otherwise, make sure this node has each of the |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1851 # required values |
|
4466
f1fe6fd0aa61
Multilinks can be filtered by combining elements with AND, OR and NOT now.
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4447
diff
changeset
|
1852 expr = Expression(v) |
|
6396
75a53956cf13
Multilink expressions with simple "or"
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6352
diff
changeset
|
1853 if expr.evaluate(nv): |
|
75a53956cf13
Multilink expressions with simple "or"
Ralf Schlatterbeck <rsc@runtux.com>
parents:
6352
diff
changeset
|
1854 match = 1 |
| 924 | 1855 elif t == STRING: |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1856 if nv is None: |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1857 nv = '' |
|
5867
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1858 if is_us(v): |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1859 # Exact match |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1860 match = (nv == v) |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1861 else: |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1862 # RE search |
|
ee2e8f8d6648
Implement exact string search
Ralf Schlatterbeck <rsc@runtux.com>
parents:
5809
diff
changeset
|
1863 match = v.search(nv) |
| 6118 | 1864 elif t == DATE: |
| 1865 for x in v: | |
| 1866 if x is None or nv is None: | |
| 1867 if nv is None and x is None: | |
| 1868 match = 1 | |
| 1869 break | |
| 1870 continue | |
| 1871 elif x.to_value: | |
| 1872 if x.from_value <= nv <= x.to_value: | |
| 1873 match = 1 | |
| 1874 break | |
| 1875 else: | |
| 1876 if x.from_value <= nv: | |
| 1877 match = 1 | |
| 1878 break | |
| 1879 elif t == INTERVAL: | |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1880 if nv is None: |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1881 match = v is None |
|
1499
8ee69708da0c
added support for searching on ranges of dates
Andrey Lebedev <kedder@users.sourceforge.net>
parents:
1496
diff
changeset
|
1882 else: |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1883 if v.to_value: |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1884 if v.from_value <= nv and v.to_value >= nv: |
|
2617
33fffbf7ae68
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2608
diff
changeset
|
1885 match = 1 |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1886 else: |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1887 if v.from_value <= nv: |
|
2617
33fffbf7ae68
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2608
diff
changeset
|
1888 match = 1 |
| 924 | 1889 elif t == OTHER: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1890 # straight value comparison for the other types |
|
2486
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1891 match = nv in v |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1892 if not match: |
|
d7488e349a03
merge from maint-0-7
Richard Jones <richard@users.sourceforge.net>
parents:
2453
diff
changeset
|
1893 break |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1894 else: |
|
2239
c8a06e10e2c6
much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
1895 matches.append([nodeid, node]) |
|
2252
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1896 |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1897 # filter based on full text search |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1898 if search_matches is not None: |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1899 k = [] |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1900 for v in matches: |
| 4044 | 1901 if v[0] in search_matches: |
|
2252
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1902 k.append(v) |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1903 matches = k |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1904 |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1905 # add sorting information to the proptree |
|
2437
f37f3617b9e9
force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents:
2418
diff
changeset
|
1906 JPROPS = {'actor':1, 'activity':1, 'creator':1, 'creation':1} |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1907 children = [] |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1908 if proptree: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1909 children = proptree.sortable_children() |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1910 for pt in children: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1911 dir = pt.sort_direction |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1912 prop = pt.name |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1913 assert (dir and prop) |
|
2252
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1914 propclass = props[prop] |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1915 pt.sort_ids = [] |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1916 is_pointer = isinstance(propclass,(hyperdb.Link, |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1917 hyperdb.Multilink)) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1918 if not is_pointer: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1919 pt.sort_result = [] |
|
2252
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1920 try: |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1921 # cache the opened link class db, if needed. |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1922 lcldb = None |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1923 # cache the linked class items too |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1924 lcache = {} |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1925 |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1926 for entry in matches: |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1927 itemid = entry[-2] |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1928 item = entry[-1] |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1929 # handle the properties that might be "faked" |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1930 # also, handle possible missing properties |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1931 try: |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1932 v = item[prop] |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1933 except KeyError: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1934 if prop in JPROPS: |
|
2437
f37f3617b9e9
force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents:
2418
diff
changeset
|
1935 # force lookup of the special journal prop |
|
f37f3617b9e9
force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents:
2418
diff
changeset
|
1936 v = self.get(itemid, prop) |
|
f37f3617b9e9
force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents:
2418
diff
changeset
|
1937 else: |
|
f37f3617b9e9
force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents:
2418
diff
changeset
|
1938 # the node doesn't have a value for this |
|
f37f3617b9e9
force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents:
2418
diff
changeset
|
1939 # property |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1940 v = None |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1941 if isinstance(propclass, hyperdb.Multilink): |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1942 v = [] |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1943 if prop == 'id': |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1944 v = int (itemid) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1945 pt.sort_ids.append(v) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1946 if not is_pointer: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1947 pt.sort_result.append(v) |
|
2437
f37f3617b9e9
force lookup of journal props in anydbm filtering
Richard Jones <richard@users.sourceforge.net>
parents:
2418
diff
changeset
|
1948 continue |
|
2252
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1949 |
|
2285
c7f780c24a87
fixed anydbm sorting with None values [SF#952853]
Richard Jones <richard@users.sourceforge.net>
parents:
2252
diff
changeset
|
1950 # missing (None) values are always sorted first |
|
c7f780c24a87
fixed anydbm sorting with None values [SF#952853]
Richard Jones <richard@users.sourceforge.net>
parents:
2252
diff
changeset
|
1951 if v is None: |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1952 pt.sort_ids.append(v) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1953 if not is_pointer: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1954 pt.sort_result.append(v) |
|
2285
c7f780c24a87
fixed anydbm sorting with None values [SF#952853]
Richard Jones <richard@users.sourceforge.net>
parents:
2252
diff
changeset
|
1955 continue |
|
c7f780c24a87
fixed anydbm sorting with None values [SF#952853]
Richard Jones <richard@users.sourceforge.net>
parents:
2252
diff
changeset
|
1956 |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1957 if isinstance(propclass, hyperdb.Link): |
|
2252
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1958 lcn = propclass.classname |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1959 link = self.db.classes[lcn] |
|
3476
1142dafe0d7f
added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
1960 key = link.orderprop() |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1961 child = pt.propdict[key] |
|
3476
1142dafe0d7f
added setorderprop() and setlabelprop() to Class (lots of patches)
Richard Jones <richard@users.sourceforge.net>
parents:
3455
diff
changeset
|
1962 if key!='id': |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
1963 if v not in lcache: |
|
2252
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1964 # open the link class db if it's not already |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1965 if lcldb is None: |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1966 lcldb = self.db.getclassdb(lcn) |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1967 lcache[v] = self.db.getnode(lcn, v, lcldb) |
|
3682
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1968 r = lcache[v][key] |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1969 child.propdict[key].sort_ids.append(r) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1970 else: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1971 child.propdict[key].sort_ids.append(v) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1972 pt.sort_ids.append(v) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1973 if not is_pointer: |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1974 r = propclass.sort_repr(pt.parent.cls, v, pt.name) |
|
193f316dbbe9
More transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3634
diff
changeset
|
1975 pt.sort_result.append(r) |
|
2252
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1976 finally: |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1977 # if we opened the link class db, close it now |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1978 if lcldb is not None: |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1979 lcldb.close() |
|
b63e455462ee
some small performance boost for *dbm backends... bsddb3 bites though
Richard Jones <richard@users.sourceforge.net>
parents:
2249
diff
changeset
|
1980 del lcache |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1981 finally: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1982 cldb.close() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1983 |
|
2239
c8a06e10e2c6
much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
1984 # pull the id out of the individual entries |
|
c8a06e10e2c6
much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
1985 matches = [entry[-2] for entry in matches] |
|
2237
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
1986 if __debug__: |
|
f624fc20f8fe
added capturing of stats
Richard Jones <richard@users.sourceforge.net>
parents:
2191
diff
changeset
|
1987 self.db.stats['filtering'] += (time.time() - start_t) |
|
2239
c8a06e10e2c6
much faster anydbm filter(), but it breaks most filtering tests
Richard Jones <richard@users.sourceforge.net>
parents:
2237
diff
changeset
|
1988 return matches |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1989 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1990 def count(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1991 """Get the number of nodes in this class. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1992 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1993 If the returned integer is 'numnodes', the ids of all the nodes |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1994 in this class run from 1 to numnodes, and numnodes+1 will be the |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1995 id of the next node to be created in this class. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
1996 """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1997 return self.db.countnodes(self.classname) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1998 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
1999 # Manipulating properties: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2000 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2001 def getprops(self, protected=1): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2002 """Return a dictionary mapping property names to property objects. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2003 If the "protected" flag is true, we include protected properties - |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2004 those which may not be modified. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2005 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2006 In addition to the actual properties on the node, these |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2007 methods provide the "creation" and "activity" properties. If the |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2008 "protected" flag is true, we include protected properties - those |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2009 which may not be modified. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2010 """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2011 d = self.properties.copy() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2012 if protected: |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
2013 d['id'] = hyperdb.String() |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2014 d['creation'] = hyperdb.Date() |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2015 d['activity'] = hyperdb.Date() |
|
1176
bd3b57859c37
On second thought, that last checkin was dumb.
Richard Jones <richard@users.sourceforge.net>
parents:
1174
diff
changeset
|
2016 d['creator'] = hyperdb.Link('user') |
|
2077
3e0961d6d44d
Added the "actor" property.
Richard Jones <richard@users.sourceforge.net>
parents:
2076
diff
changeset
|
2017 d['actor'] = hyperdb.Link('user') |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2018 return d |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2019 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2020 def addprop(self, **properties): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2021 """Add properties to this class. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2022 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2023 The keyword arguments in 'properties' must map names to property |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2024 objects, or a TypeError is raised. None of the keys in 'properties' |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2025 may collide with the names of existing properties, or a ValueError |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2026 is raised before any properties have been added. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2027 """ |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2028 for key in properties: |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2029 if key in self.properties: |
|
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2030 raise ValueError(key) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2031 self.properties.update(properties) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2032 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2033 def index(self, nodeid): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2034 """ Add (or refresh) the node to search indexes """ |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2035 # find all the String properties that have indexme |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
2036 for prop, propclass in self.getprops().items(): |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2037 if isinstance(propclass, hyperdb.String) and propclass.indexme: |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2038 # index them under (classname, nodeid, property) |
|
891
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
2039 try: |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
2040 value = str(self.get(nodeid, prop)) |
|
974a4b94c5e3
Implemented the destroy() method needed by the session database...
Richard Jones <richard@users.sourceforge.net>
parents:
890
diff
changeset
|
2041 except IndexError: |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2042 # node has been destroyed |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2043 continue |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2044 self.db.indexer.add_text((self.classname, nodeid, prop), value) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2045 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2046 # |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2047 # import / export support |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2048 # |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2049 def export_list(self, propnames, nodeid): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2050 """ Export a node - generate a list of CSV-able data in the order |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2051 specified by propnames for the given node. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2052 """ |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2053 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2054 l = [] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2055 for prop in propnames: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2056 proptype = properties[prop] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2057 value = self.get(nodeid, prop) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2058 # "marshal" data where needed |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2059 if value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2060 pass |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2061 elif isinstance(proptype, hyperdb.Date): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2062 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2063 elif isinstance(proptype, hyperdb.Interval): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2064 value = value.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2065 elif isinstance(proptype, hyperdb.Password): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2066 value = str(value) |
|
5525
bb7865241f8a
Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents:
5492
diff
changeset
|
2067 l.append(repr_export(value)) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2068 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2069 # append retired flag |
|
5525
bb7865241f8a
Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents:
5492
diff
changeset
|
2070 l.append(repr_export(self.is_retired(nodeid))) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2071 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2072 return l |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2073 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2074 def import_list(self, propnames, proplist): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2075 """ Import a node - all information including "id" is present and |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2076 should not be sanity checked. Triggers are not triggered. The |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2077 journal should be initialised using the "creator" and "created" |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2078 information. |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2079 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2080 Return the nodeid of the node imported. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2081 """ |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2082 if self.db.journaltag is None: |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2083 raise hyperdb.DatabaseError(_('Database open read-only')) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2084 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2085 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2086 # make the new node's property map |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2087 d = {} |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2088 newid = None |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2089 for i in range(len(propnames)): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2090 # Figure the property for this column |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2091 propname = propnames[i] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2092 |
|
5525
bb7865241f8a
Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents:
5492
diff
changeset
|
2093 # Use eval_import to reverse the repr_export() used to |
|
bb7865241f8a
Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents:
5492
diff
changeset
|
2094 # output the CSV |
|
bb7865241f8a
Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents:
5492
diff
changeset
|
2095 value = eval_import(proplist[i]) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2096 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2097 # "unmarshal" where necessary |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2098 if propname == 'id': |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2099 newid = value |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2100 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2101 elif propname == 'is retired': |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2102 # is the item retired? |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2103 if int(value): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2104 d[self.db.RETIRED_FLAG] = 1 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2105 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2106 elif value is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2107 d[propname] = None |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2108 continue |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2109 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2110 prop = properties[propname] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2111 if isinstance(prop, hyperdb.Date): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2112 value = date.Date(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2113 elif isinstance(prop, hyperdb.Interval): |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2114 value = date.Interval(value) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2115 elif isinstance(prop, hyperdb.Password): |
|
4480
1613754d2646
Fix first part of Password handling security issue2550688
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
4466
diff
changeset
|
2116 value = password.Password(encrypted=value) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2117 d[propname] = value |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2118 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2119 # get a new id if necessary |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2120 if newid is None: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2121 newid = self.db.newid(self.classname) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2122 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2123 # add the node and journal |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2124 self.db.addnode(self.classname, newid, d) |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2125 return newid |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2126 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2127 def export_journals(self): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2128 """Export a class's journal - generate a list of lists of |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2129 CSV-able data: |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2130 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2131 nodeid, date, user, action, params |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2132 |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2133 No heading here - the columns are fixed. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2134 """ |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2135 properties = self.getprops() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2136 r = [] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2137 for nodeid in self.getnodeids(): |
|
5232
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
2138 for nodeid, date, user, action, params in self.history(nodeid, |
|
462b0f76fce8
issue2550864 - Potential information leakage via journal/history
John Rouillard <rouilj@ieee.org>
parents:
5112
diff
changeset
|
2139 enforceperm=False, skipquiet=False): |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2140 date = date.get_tuple() |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2141 if action == 'set': |
|
3003
855e9f99c96c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
2966
diff
changeset
|
2142 export_data = {} |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
2143 for propname, value in params.items(): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2144 if propname not in properties: |
|
3003
855e9f99c96c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
2966
diff
changeset
|
2145 # property no longer in the schema |
|
855e9f99c96c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
2966
diff
changeset
|
2146 continue |
|
855e9f99c96c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
2966
diff
changeset
|
2147 |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2148 prop = properties[propname] |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2149 # make sure the params are eval()'able |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2150 if value is None: |
|
3826
bf2e9535da00
Journal and database testing.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents:
3687
diff
changeset
|
2151 pass |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
2152 elif isinstance(prop, hyperdb.Date): |
|
3166
3204488d20b5
hack to fix some anydbm export problems [SF#1081454]
Richard Jones <richard@users.sourceforge.net>
parents:
3155
diff
changeset
|
2153 # this is a hack - some dates are stored as strings |
|
3204488d20b5
hack to fix some anydbm export problems [SF#1081454]
Richard Jones <richard@users.sourceforge.net>
parents:
3155
diff
changeset
|
2154 if not isinstance(value, type('')): |
|
3204488d20b5
hack to fix some anydbm export problems [SF#1081454]
Richard Jones <richard@users.sourceforge.net>
parents:
3155
diff
changeset
|
2155 value = value.get_tuple() |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
2156 elif isinstance(prop, hyperdb.Interval): |
|
3390
a2e4b8ab5b51
include hack from [SF#1238571]
Richard Jones <richard@users.sourceforge.net>
parents:
3383
diff
changeset
|
2157 # hack too - some intervals are stored as strings |
|
a2e4b8ab5b51
include hack from [SF#1238571]
Richard Jones <richard@users.sourceforge.net>
parents:
3383
diff
changeset
|
2158 if not isinstance(value, type('')): |
|
a2e4b8ab5b51
include hack from [SF#1238571]
Richard Jones <richard@users.sourceforge.net>
parents:
3383
diff
changeset
|
2159 value = value.get_tuple() |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
2160 elif isinstance(prop, hyperdb.Password): |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2161 value = str(value) |
|
3003
855e9f99c96c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
2966
diff
changeset
|
2162 export_data[propname] = value |
| 3052 | 2163 params = export_data |
|
5525
bb7865241f8a
Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents:
5492
diff
changeset
|
2164 r.append([repr_export(nodeid), repr_export(date), |
|
bb7865241f8a
Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents:
5492
diff
changeset
|
2165 repr_export(user), repr_export(action), |
|
bb7865241f8a
Make CSV import/export compatible across Python versions (also RDBMS journals) (issue 2550976, issue 2550975).
Joseph Myers <jsm@polyomino.org.uk>
parents:
5492
diff
changeset
|
2166 repr_export(params)]) |
|
2175
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2167 return r |
|
723098a10677
Export and import now include journals (incompatible with export < 0.7)
Richard Jones <richard@users.sourceforge.net>
parents:
2089
diff
changeset
|
2168 |
|
2597
c86b2179085b
fix journal export of files to remove content from CSV files
Richard Jones <richard@users.sourceforge.net>
parents:
2514
diff
changeset
|
2169 class FileClass(hyperdb.FileClass, Class): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2170 """This class defines a large chunk of data. To support this, it has a |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2171 mandatory String property "content" which is typically saved off |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2172 externally to the hyperdb. |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2173 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2174 The default MIME type of this data is defined by the |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2175 "default_mime_type" class attribute, which may be overridden by each |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2176 node if the class defines a "type" String property. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2177 """ |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2178 def __init__(self, db, classname, **properties): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2179 """The newly-created class automatically includes the "content" |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2180 and "type" properties. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2181 """ |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2182 if 'content' not in properties: |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2183 properties['content'] = hyperdb.String(indexme='yes') |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2184 if 'type' not in properties: |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2185 properties['type'] = hyperdb.String() |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2186 Class.__init__(self, db, classname, **properties) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2187 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2188 def create(self, **propvalues): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2189 """ Snarf the "content" propvalue and store in a file |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2190 """ |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2191 # we need to fire the auditors now, or the content property won't |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2192 # be in propvalues for the auditors to play with |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2193 self.fireAuditors('create', None, propvalues) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2194 |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2195 # now remove the content property so it's not stored in the db |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2196 content = propvalues['content'] |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2197 del propvalues['content'] |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2198 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2199 # make sure we have a MIME type |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2200 mime_type = propvalues.get('type', self.default_mime_type) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2201 |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2202 # do the database create |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2203 newid = self.create_inner(**propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2204 |
|
3979
954971d612ee
Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents:
3960
diff
changeset
|
2205 # store off the content as a file |
|
5492
6b0c542642be
blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5467
diff
changeset
|
2206 self.db.storefile(self.classname, newid, None, bs2b(content)) |
|
3979
954971d612ee
Fire reactors after file storage is all done (patch [SF#2001243])
Richard Jones <richard@users.sourceforge.net>
parents:
3960
diff
changeset
|
2207 |
|
1431
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2208 # fire reactors |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2209 self.fireReactors('create', newid, None) |
|
c70068162e64
Altered Class.create() and FileClass.create() methods...
Richard Jones <richard@users.sourceforge.net>
parents:
1424
diff
changeset
|
2210 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2211 return newid |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2212 |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2213 def get(self, nodeid, propname, default=_marker, cache=1): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2214 """ Trap the content propname and get it from the file |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
2215 |
|
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
2216 'cache' exists for backwards compatibility, and is not used. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2217 """ |
|
1333
80d27b7d6db5
implemented whole-database locking
Richard Jones <richard@users.sourceforge.net>
parents:
1327
diff
changeset
|
2218 poss_msg = 'Possibly an access right configuration problem.' |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2219 if propname == 'content': |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2220 try: |
|
5492
6b0c542642be
blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5467
diff
changeset
|
2221 return b2s(self.db.getfile(self.classname, nodeid, None)) |
|
5248
198b6e810c67
Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents:
5232
diff
changeset
|
2222 except IOError as strerror: |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
2223 # XXX by catching this we don't see an error in the log. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2224 return 'ERROR reading file: %s%s\n%s\n%s'%( |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2225 self.classname, nodeid, poss_msg, strerror) |
|
5725
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
2226 except UnicodeDecodeError as e: |
|
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
2227 # if content is not text (e.g. jpeg file) we get |
|
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
2228 # unicode error trying to convert to string in python 3. |
|
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
2229 # trap it and supply an error message. Include md5sum |
|
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
2230 # of content as this string is included in the etag |
|
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
2231 # calculation of the object. |
|
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
2232 return ('%s%s is not text, retrieve using ' |
|
6923225fd781
Handle UnicodeDecodeError in file class when file contents are not
John Rouillard <rouilj@ieee.org>
parents:
5704
diff
changeset
|
2233 'binary_content property. mdsum: %s')%(self.classname, |
| 6002 | 2234 nodeid, md5(self.db.getfile(self.classname, nodeid, None)).hexdigest()) # nosec - bandit md5 use ok |
|
5492
6b0c542642be
blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5467
diff
changeset
|
2235 elif propname == 'binary_content': |
|
6b0c542642be
blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5467
diff
changeset
|
2236 return self.db.getfile(self.classname, nodeid, None) |
|
6b0c542642be
blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5467
diff
changeset
|
2237 |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2238 if default is not _marker: |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
2239 return Class.get(self, nodeid, propname, default) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2240 else: |
|
1780
d2801a2b0a77
Initial implementation (half-baked) at new Tracker instance.
Richard Jones <richard@users.sourceforge.net>
parents:
1751
diff
changeset
|
2241 return Class.get(self, nodeid, propname) |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2242 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2243 def set(self, itemid, **propvalues): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2244 """ Snarf the "content" propvalue and update it in a file |
|
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2245 """ |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2246 self.fireAuditors('set', itemid, propvalues) |
|
2608
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
2247 |
|
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
2248 # create the oldvalues dict - fill in any missing values |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2249 oldvalues = copy.deepcopy(self.db.getnode(self.classname, itemid)) |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
2250 for name, prop in self.getprops(protected=0).items(): |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2251 if name in oldvalues: |
|
2608
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
2252 continue |
|
2906
a8808157f892
fix some bugs introduced in refactoring of blobfiles filename()
Richard Jones <richard@users.sourceforge.net>
parents:
2899
diff
changeset
|
2253 if isinstance(prop, hyperdb.Multilink): |
|
2608
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
2254 oldvalues[name] = [] |
|
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
2255 else: |
|
151ee7f0ca7d
fix another bug exposed by earlier change to defaulting of *dbm property values
Richard Jones <richard@users.sourceforge.net>
parents:
2606
diff
changeset
|
2256 oldvalues[name] = None |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2257 |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2258 # now remove the content property so it's not stored in the db |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2259 content = None |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2260 if 'content' in propvalues: |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2261 content = propvalues['content'] |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2262 del propvalues['content'] |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2263 |
|
2640
b01eca163779
The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents:
2637
diff
changeset
|
2264 # do the database update |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2265 propvalues = self.set_inner(itemid, **propvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2266 |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2267 # do content? |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2268 if content: |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2269 # store and possibly index |
|
5492
6b0c542642be
blobfiles now always stores/returns bytes
Christof Meerwald <cmeerw@cmeerw.org>
parents:
5467
diff
changeset
|
2270 self.db.storefile(self.classname, itemid, None, bs2b(content)) |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2271 if self.properties['content'].indexme: |
|
5544
1a0498c1ed90
Avoid errors indexing binary uploads with Python 3.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5525
diff
changeset
|
2272 index_content = content |
|
1a0498c1ed90
Avoid errors indexing binary uploads with Python 3.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5525
diff
changeset
|
2273 if bytes != str and isinstance(content, bytes): |
|
1a0498c1ed90
Avoid errors indexing binary uploads with Python 3.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5525
diff
changeset
|
2274 index_content = content.decode('utf-8', errors='ignore') |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2275 mime_type = self.get(itemid, 'type', self.default_mime_type) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2276 self.db.indexer.add_text((self.classname, itemid, 'content'), |
|
5544
1a0498c1ed90
Avoid errors indexing binary uploads with Python 3.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5525
diff
changeset
|
2277 index_content, mime_type) |
|
2640
b01eca163779
The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents:
2637
diff
changeset
|
2278 propvalues['content'] = content |
|
b01eca163779
The "type" parameter is supposed to be optional
Richard Jones <richard@users.sourceforge.net>
parents:
2637
diff
changeset
|
2279 |
|
2089
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2280 # fire reactors |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2281 self.fireReactors('set', itemid, oldvalues) |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2282 return propvalues |
|
93f03c6714d8
A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents:
2082
diff
changeset
|
2283 |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2284 def index(self, nodeid): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2285 """ Add (or refresh) the node to search indexes. |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2286 |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2287 Use the content-type property for the content property. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2288 """ |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2289 # find all the String properties that have indexme |
|
5395
23b8e6067f7c
Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5378
diff
changeset
|
2290 for prop, propclass in self.getprops().items(): |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2291 if prop == 'content' and propclass.indexme: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2292 mime_type = self.get(nodeid, 'type', self.default_mime_type) |
|
5544
1a0498c1ed90
Avoid errors indexing binary uploads with Python 3.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5525
diff
changeset
|
2293 index_content = self.get(nodeid, 'binary_content') |
|
1a0498c1ed90
Avoid errors indexing binary uploads with Python 3.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5525
diff
changeset
|
2294 if bytes != str and isinstance(index_content, bytes): |
|
1a0498c1ed90
Avoid errors indexing binary uploads with Python 3.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5525
diff
changeset
|
2295 index_content = index_content.decode('utf-8', |
|
1a0498c1ed90
Avoid errors indexing binary uploads with Python 3.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5525
diff
changeset
|
2296 errors='ignore') |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2297 self.db.indexer.add_text((self.classname, nodeid, 'content'), |
|
5544
1a0498c1ed90
Avoid errors indexing binary uploads with Python 3.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5525
diff
changeset
|
2298 index_content, mime_type) |
|
3601
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2299 elif isinstance(propclass, hyperdb.String) and propclass.indexme: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2300 # index them under (classname, nodeid, property) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2301 try: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2302 value = str(self.get(nodeid, prop)) |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2303 except IndexError: |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2304 # node has been destroyed |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2305 continue |
|
7b25567f0f54
indexing may be turned off for FileClass "content" now
Richard Jones <richard@users.sourceforge.net>
parents:
3589
diff
changeset
|
2306 self.db.indexer.add_text((self.classname, nodeid, prop), value) |
|
2505
bdd112cf61ba
rdbms backend full text search failure after import [SF#980314]
Richard Jones <richard@users.sourceforge.net>
parents:
2496
diff
changeset
|
2307 |
|
1103
db787cef1385
handled some XXXs
Richard Jones <richard@users.sourceforge.net>
parents:
1099
diff
changeset
|
2308 # deviation from spec - was called ItemClass |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2309 class IssueClass(Class, roundupdb.IssueClass): |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2310 # Overridden methods: |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2311 def __init__(self, db, classname, **properties): |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2312 """The newly-created class automatically includes the "messages", |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2313 "files", "nosy", and "superseder" properties. If the 'properties' |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2314 dictionary attempts to specify any of these properties or a |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2315 "creation" or "activity" property, a ValueError is raised. |
|
4075
b87d022a2f40
Uniformly use """...""" instead of '''...''' for comments.
Stefan Seefeld <stefan@seefeld.name>
parents:
4044
diff
changeset
|
2316 """ |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2317 if 'title' not in properties: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2318 properties['title'] = hyperdb.String(indexme='yes') |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2319 if 'messages' not in properties: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2320 properties['messages'] = hyperdb.Multilink("msg") |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2321 if 'files' not in properties: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2322 properties['files'] = hyperdb.Multilink("file") |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2323 if 'nosy' not in properties: |
| 1002 | 2324 # note: journalling is turned off as it really just wastes |
| 2325 # space. this behaviour may be overridden in an instance | |
| 2326 properties['nosy'] = hyperdb.Multilink("user", do_journal="no") | |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4348
diff
changeset
|
2327 if 'superseder' not in properties: |
|
858
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2328 properties['superseder'] = hyperdb.Multilink(classname) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2329 Class.__init__(self, db, classname, **properties) |
|
2dd862af72ee
all storage-specific code (ie. backend) is now implemented by the backends
Richard Jones <richard@users.sourceforge.net>
parents:
843
diff
changeset
|
2330 |
|
2699
2f5bf63a4b2c
trim trailing spaces; add vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
2650
diff
changeset
|
2331 # vim: set et sts=4 sw=4 : |
