annotate roundup/backends/sessions_dbm.py @ 8241:741ea8a86012

fix: issue2551374. Error handling for filter expressions. Errors in filter expressions are now reported. The UI needs some work but even the current code is helpful when debugging filter expressions. mlink_expr: defines/raises ExpressionError(error string template, context=dict()) raises ExpressionError when it detects errors when popping arguments off stack raises ExpressionError when more than one element left on the stack before returning also ruff fix to group boolean expression with parens back_anydbm.py, rdbms_common.py: catches ExpressionError, augments context with class and attribute being searched. raises the exception for both link and multilink relations client.py catches ExpressionError returning a basic error page. The page is a dead end. There are no links or anything for the user to move forward. The user has to go back, possibly refresh the page (because the submit button may be disalbled) re-enter the query and try again. This needs to be improved. test_liveserver.py test the error page generated by client.py db_test_base unit tests for filter with too few arguments, too many arguments, check all repr and str formats.
author John Rouillard <rouilj@ieee.org>
date Mon, 30 Dec 2024 20:22:55 -0500
parents a5d39e5a5cdc
children accddaeda449
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1 """This module defines a very basic store that's used by the CGI interface
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2 to store session and one-time-key information.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
4 Yes, it's called "sessions" - because originally it only defined a session
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
5 class. It's now also used for One Time Key handling too.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
6 """
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
7 __docformat__ = 'restructuredtext'
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
8
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
9 import marshal, os, random, time
4362
74476eaac38a more modernisation
Richard Jones <richard@users.sourceforge.net>
parents: 3989
diff changeset
10
5837
883c9e90b403 Fix problem with cgi.escape being depricated a different way. This way
John Rouillard <rouilj@ieee.org>
parents: 5806
diff changeset
11 from roundup.anypy.html import html_escape as escape
5806
abee2c2c822e More cgi.escape/html.escape fixes.
John Rouillard <rouilj@ieee.org>
parents: 5732
diff changeset
12
3924
21d3d7eeea8c assorted pyflakes fixes
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3609
diff changeset
13 from roundup import hyperdb
3925
603ec9630b08 i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3924
diff changeset
14 from roundup.i18n import _
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
15 from roundup.anypy.dbm_ import anydbm, whichdb
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
16 from roundup.backends.sessions_common import SessionCommon
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
17
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
18
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
19 class BasicDatabase(SessionCommon):
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
20 ''' Provide a nice encapsulation of an anydbm store.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
21
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
22 Keys are id strings, values are automatically marshalled data.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
23 '''
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
24 _db_type = None
5252
0b154486ed38 Make sure that the name property is initialized to none.
John Rouillard <rouilj@ieee.org>
parents: 5011
diff changeset
25 name = None
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
26
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
27 def __init__(self, db):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
28 self.config = db.config
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
29 self.dir = db.config.DATABASE
3609
f2fda3e6fc8b umask is now configurable (with the same 0002 default)
Richard Jones <richard@users.sourceforge.net>
parents: 3606
diff changeset
30 os.umask(db.config.UMASK)
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
31
2169
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
32 def exists(self, infoid):
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
33 db = self.opendb('c')
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
34 try:
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
35 return infoid in db
2169
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
36 finally:
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
37 db.close()
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
38
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
39 def clear(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
40 path = os.path.join(self.dir, self.name)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
41 if os.path.exists(path):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
42 os.remove(path)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
43 elif os.path.exists(path+'.db'): # dbm appends .db
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
44 os.remove(path+'.db')
7879
39c482e6a246 fix: fix code to make tests of session and otks databases pass on windows
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
45 elif os.path.exists(path+".dir"): # dumb dbm
39c482e6a246 fix: fix code to make tests of session and otks databases pass on windows
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
46 os.remove(path+".dir")
39c482e6a246 fix: fix code to make tests of session and otks databases pass on windows
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
47 os.remove(path+".dat")
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
48
7889
a5d39e5a5cdc fix: revert setting _db_type when creating new db.
John Rouillard <rouilj@ieee.org>
parents: 7888
diff changeset
49 #self._db_type = None
7888
456edf872259 fix: set _db_type to None when clearing anydbm db.
John Rouillard <rouilj@ieee.org>
parents: 7879
diff changeset
50
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
51 def cache_db_type(self, path):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
52 ''' determine which DB wrote the class file, and cache it as an
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
53 attribute of __class__ (to allow for subclassed DBs to be
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
54 different sorts)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
55 '''
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
56 db_type = ''
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
57 if os.path.exists(path):
4363
Richard Jones <richard@users.sourceforge.net>
parents: 4362
diff changeset
58 db_type = whichdb(path)
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
59 if not db_type:
4362
74476eaac38a more modernisation
Richard Jones <richard@users.sourceforge.net>
parents: 3989
diff changeset
60 raise hyperdb.DatabaseError(
74476eaac38a more modernisation
Richard Jones <richard@users.sourceforge.net>
parents: 3989
diff changeset
61 _("Couldn't identify database type"))
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
62 elif os.path.exists(path+'.db'):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
63 # if the path ends in '.db', it's a dbm database, whether
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
64 # anydbm says it's dbhash or not!
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
65 db_type = 'dbm'
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
66 self.__class__._db_type = db_type
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
67
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
68 _marker = []
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
69
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
70 def get(self, infoid, value, default=_marker):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
71 db = self.opendb('c')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
72 try:
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
73 if infoid in db:
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
74 values = marshal.loads(db[infoid])
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
75 else:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
76 if default != self._marker:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
77 return default
6036
5eeb073e33c7 flake8 whitespace fixes, remove unused exception var.
John Rouillard <rouilj@ieee.org>
parents: 5837
diff changeset
78 raise KeyError('No such %s "%s"' % (self.name, escape(infoid)))
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
79 return values.get(value, None)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
80 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
81 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
82
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
83 def getall(self, infoid):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
84 db = self.opendb('c')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
85 try:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
86 try:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
87 d = marshal.loads(db[infoid])
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
88 del d['__timestamp']
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
89 return d
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
90 except KeyError:
6036
5eeb073e33c7 flake8 whitespace fixes, remove unused exception var.
John Rouillard <rouilj@ieee.org>
parents: 5837
diff changeset
91 raise KeyError('No such %s "%s"' % (self.name, escape(infoid)))
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
92 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
93 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
94
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
95 def set(self, infoid, **newvalues):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
96 db = self.opendb('c')
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
97 timestamp = None
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
98 try:
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
99 if infoid in db:
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
100 values = marshal.loads(db[infoid])
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
101 try:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
102 timestamp = values['__timestamp']
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
103 except KeyError:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
104 pass # stay at None
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
105 else:
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
106 values = {}
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
107
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
108 if '__timestamp' in newvalues:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
109 try:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
110 float(newvalues['__timestamp'])
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
111 except ValueError:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
112 # keep original timestamp if present
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
113 newvalues['__timestamp'] = timestamp or time.time()
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
114 else:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
115 newvalues['__timestamp'] = time.time()
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
116
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
117 values.update(newvalues)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
118 db[infoid] = marshal.dumps(values)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
119 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
120 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
121
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
122 def list(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
123 db = self.opendb('r')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
124 try:
4385
8d135c19523e fix up some pre-Python2.6 compatibility issues in the *dbm interface
Richard Jones <richard@users.sourceforge.net>
parents: 4383
diff changeset
125 return list(db.keys())
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
126 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
127 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
128
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
129 def destroy(self, infoid):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
130 db = self.opendb('c')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
131 try:
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
132 if infoid in db:
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
133 del db[infoid]
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
134 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
135 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
136
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
137 def opendb(self, mode):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
138 '''Low-level database opener that gets around anydbm/dbm
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
139 eccentricities.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
140 '''
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
141 # figure the class db type
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
142 path = os.path.join(os.getcwd(), self.dir, self.name)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
143 if self._db_type is None:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
144 self.cache_db_type(path)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
145
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
146 db_type = self._db_type
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
147
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
148 # new database? let anydbm pick the best dbm
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
149 if not db_type:
7879
39c482e6a246 fix: fix code to make tests of session and otks databases pass on windows
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
150 db = anydbm.open(path, 'c')
7889
a5d39e5a5cdc fix: revert setting _db_type when creating new db.
John Rouillard <rouilj@ieee.org>
parents: 7888
diff changeset
151 #self.cache_db_type(path)
7879
39c482e6a246 fix: fix code to make tests of session and otks databases pass on windows
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
152 return db
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
153
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
154 # open the database with the correct module
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
155 dbm = __import__(db_type)
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
156
6036
5eeb073e33c7 flake8 whitespace fixes, remove unused exception var.
John Rouillard <rouilj@ieee.org>
parents: 5837
diff changeset
157 retries_left = 15
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
158 while True:
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
159 try:
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
160 handle = dbm.open(path, mode)
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
161 break
6565
2c2dbfc332ba Try to handle multiple connections better.
John Rouillard <rouilj@ieee.org>
parents: 6064
diff changeset
162 except OSError as e:
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
163 # Primarily we want to catch and retry:
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
164 # [Errno 11] Resource temporarily unavailable retry
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
165 # FIXME: make this more specific
6565
2c2dbfc332ba Try to handle multiple connections better.
John Rouillard <rouilj@ieee.org>
parents: 6064
diff changeset
166 if retries_left < 10:
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
167 self.log_warning(
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
168 'dbm.open failed on ...%s, retry %s left: %s, %s' %
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
169 (path[-15:], 15-retries_left, retries_left, e))
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
170 if retries_left < 0:
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
171 # We have used up the retries. Reraise the exception
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
172 # that got us here.
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
173 raise
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
174 else:
6565
2c2dbfc332ba Try to handle multiple connections better.
John Rouillard <rouilj@ieee.org>
parents: 6064
diff changeset
175 # stagger retry to try to get around thundering herd issue.
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
176 time.sleep(random.randint(0, 25)*.005)
6064
bef1e42be04c Flake8 whitespace changes.
John Rouillard <rouilj@ieee.org>
parents: 6036
diff changeset
177 retries_left = retries_left - 1
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
178 continue # the while loop
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
179 return handle
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
180
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
181 def commit(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
182 pass
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
183
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
184 def lifetime(self, key_lifetime=0):
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
185 """Return the proper timestamp for a key with key_lifetime specified
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
186 in seconds. Default lifetime is 0.
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
187 """
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
188 now = time.time()
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
189 week = 60*60*24*7
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
190 return now - week + key_lifetime
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
191
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
192 def close(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
193 pass
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
194
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
195 def updateTimestamp(self, sessid):
3606
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
196 ''' don't update every hit - once a minute should be OK '''
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
197 sess = self.get(sessid, '__timestamp', None)
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
198 now = time.time()
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
199 if sess is None or now > sess + 60:
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
200 self.set(sessid, __timestamp=now)
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
201
3989
0112e9e1d068 improvements to session management
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
202 def clean(self):
0112e9e1d068 improvements to session management
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
203 ''' Remove session records that haven't been used for a week. '''
0112e9e1d068 improvements to session management
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
204 now = time.time()
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
205 week = 60*60*24*7
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
206 for sessid in self.list():
2151
ae21c1fc41d5 better fix
Richard Jones <richard@users.sourceforge.net>
parents: 2150
diff changeset
207 sess = self.get(sessid, '__timestamp', None)
ae21c1fc41d5 better fix
Richard Jones <richard@users.sourceforge.net>
parents: 2150
diff changeset
208 if sess is None:
ae21c1fc41d5 better fix
Richard Jones <richard@users.sourceforge.net>
parents: 2150
diff changeset
209 self.updateTimestamp(sessid)
3989
0112e9e1d068 improvements to session management
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
210 continue
2151
ae21c1fc41d5 better fix
Richard Jones <richard@users.sourceforge.net>
parents: 2150
diff changeset
211 interval = now - sess
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
212 if interval > week:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
213 self.destroy(sessid)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
214
6064
bef1e42be04c Flake8 whitespace changes.
John Rouillard <rouilj@ieee.org>
parents: 6036
diff changeset
215
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
216 class Sessions(BasicDatabase):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
217 name = 'sessions'
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
218
6064
bef1e42be04c Flake8 whitespace changes.
John Rouillard <rouilj@ieee.org>
parents: 6036
diff changeset
219
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
220 class OneTimeKeys(BasicDatabase):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
221 name = 'otks'
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
222
3925
603ec9630b08 i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3924
diff changeset
223 # vim: set sts ts=4 sw=4 et si :

Roundup Issue Tracker: http://roundup-tracker.org/