Mercurial > p > roundup > code

annotate roundup/backends/sessions_dbm.py @ 6808:375d40a9e730

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add tests to BasicDatabase and merge implementations test/session_common.py: add new tests: * test set with bad __timestamps * test get on missing item with no default * test clear() method * test new lifetime() method everything below here was needed to get the tests to work across all db's. roundup/backends/sessions_dbm.py: make set() validate __timestamp as float. If invalid and item is new, set it to time.time(). If invalid and item exists keep original timestamp. add lifetime(key_lifetime) method. Given key_lifetime in seconds, generate a __timestamp that will be deleted after that many seconds. roundup/backends/sessions_rdbms.py: make set() behave the same as session_dbm. add lifetime method as above. roundup/backends/sessions_sqlite.py: import session_rdbms::BasicDatabase and override __init__. rather than cloning all the code. Kept a few logging and sql methods. roundup/test/memorydb.py: make get() on a missing key return KeyError make set() conform to dbm/rdbms implementations.
author John Rouillard <rouilj@ieee.org>
date Mon, 25 Jul 2022 21:21:26 -0400
parents b4d0b48b3096
children fe0091279f50
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1 """This module defines a very basic store that's used by the CGI interface
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2 to store session and one-time-key information.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
4 Yes, it's called "sessions" - because originally it only defined a session
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
5 class. It's now also used for One Time Key handling too.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
6 """
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
7 __docformat__ = 'restructuredtext'
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
8
6565
2c2dbfc332ba Try to handle multiple connections better.
John Rouillard <rouilj@ieee.org>
parents: 6064
diff changeset
9 import os, marshal, time, logging, random
4362
74476eaac38a more modernisation
Richard Jones <richard@users.sourceforge.net>
parents: 3989
diff changeset
10
5837
883c9e90b403 Fix problem with cgi.escape being depricated a different way. This way
John Rouillard <rouilj@ieee.org>
parents: 5806
diff changeset
11 from roundup.anypy.html import html_escape as escape
5806
abee2c2c822e More cgi.escape/html.escape fixes.
John Rouillard <rouilj@ieee.org>
parents: 5732
diff changeset
12
3924
21d3d7eeea8c assorted pyflakes fixes
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3609
diff changeset
13 from roundup import hyperdb
3925
603ec9630b08 i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3924
diff changeset
14 from roundup.i18n import _
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
15 from roundup.anypy.dbm_ import anydbm, whichdb
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
16
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
17
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
18 class BasicDatabase:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
19 ''' Provide a nice encapsulation of an anydbm store.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
20
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
21 Keys are id strings, values are automatically marshalled data.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
22 '''
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
23 _db_type = None
5252
0b154486ed38 Make sure that the name property is initialized to none.
John Rouillard <rouilj@ieee.org>
parents: 5011
diff changeset
24 name = None
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
25
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
26 def __init__(self, db):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
27 self.config = db.config
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
28 self.dir = db.config.DATABASE
3609
f2fda3e6fc8b umask is now configurable (with the same 0002 default)
Richard Jones <richard@users.sourceforge.net>
parents: 3606
diff changeset
29 os.umask(db.config.UMASK)
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
30
2169
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
31 def exists(self, infoid):
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
32 db = self.opendb('c')
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
33 try:
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
34 return infoid in db
2169
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
35 finally:
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
36 db.close()
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
37
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
38 def clear(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
39 path = os.path.join(self.dir, self.name)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
40 if os.path.exists(path):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
41 os.remove(path)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
42 elif os.path.exists(path+'.db'): # dbm appends .db
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
43 os.remove(path+'.db')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
44
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
45 def cache_db_type(self, path):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
46 ''' determine which DB wrote the class file, and cache it as an
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
47 attribute of __class__ (to allow for subclassed DBs to be
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
48 different sorts)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
49 '''
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
50 db_type = ''
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
51 if os.path.exists(path):
4363
Richard Jones <richard@users.sourceforge.net>
parents: 4362
diff changeset
52 db_type = whichdb(path)
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
53 if not db_type:
4362
74476eaac38a more modernisation
Richard Jones <richard@users.sourceforge.net>
parents: 3989
diff changeset
54 raise hyperdb.DatabaseError(
74476eaac38a more modernisation
Richard Jones <richard@users.sourceforge.net>
parents: 3989
diff changeset
55 _("Couldn't identify database type"))
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
56 elif os.path.exists(path+'.db'):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
57 # if the path ends in '.db', it's a dbm database, whether
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
58 # anydbm says it's dbhash or not!
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
59 db_type = 'dbm'
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
60 self.__class__._db_type = db_type
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
61
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
62 _marker = []
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
63
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
64 def get(self, infoid, value, default=_marker):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
65 db = self.opendb('c')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
66 try:
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
67 if infoid in db:
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
68 values = marshal.loads(db[infoid])
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
69 else:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
70 if default != self._marker:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
71 return default
6036
5eeb073e33c7 flake8 whitespace fixes, remove unused exception var.
John Rouillard <rouilj@ieee.org>
parents: 5837
diff changeset
72 raise KeyError('No such %s "%s"' % (self.name, escape(infoid)))
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
73 return values.get(value, None)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
74 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
75 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
76
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
77 def getall(self, infoid):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
78 db = self.opendb('c')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
79 try:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
80 try:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
81 d = marshal.loads(db[infoid])
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
82 del d['__timestamp']
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
83 return d
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
84 except KeyError:
6036
5eeb073e33c7 flake8 whitespace fixes, remove unused exception var.
John Rouillard <rouilj@ieee.org>
parents: 5837
diff changeset
85 raise KeyError('No such %s "%s"' % (self.name, escape(infoid)))
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
86 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
87 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
88
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
89 def set(self, infoid, **newvalues):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
90 db = self.opendb('c')
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
91 timestamp=None
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
92 try:
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
93 if infoid in db:
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
94 values = marshal.loads(db[infoid])
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
95 try:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
96 timestamp = values['__timestamp']
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
97 except KeyError:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
98 pass # stay at None
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
99 else:
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
100 values = {}
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
101
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
102 if '__timestamp' in newvalues:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
103 try:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
104 float(newvalues['__timestamp'])
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
105 except ValueError:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
106 # keep original timestamp if present
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
107 newvalues['__timestamp'] = timestamp or time.time()
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
108 else:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
109 newvalues['__timestamp'] = time.time()
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
110
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
111 values.update(newvalues)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
112 db[infoid] = marshal.dumps(values)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
113 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
114 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
115
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
116 def list(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
117 db = self.opendb('r')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
118 try:
4385
8d135c19523e fix up some pre-Python2.6 compatibility issues in the *dbm interface
Richard Jones <richard@users.sourceforge.net>
parents: 4383
diff changeset
119 return list(db.keys())
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
120 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
121 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
122
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
123 def destroy(self, infoid):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
124 db = self.opendb('c')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
125 try:
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
126 if infoid in db:
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
127 del db[infoid]
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
128 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
129 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
130
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
131 def opendb(self, mode):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
132 '''Low-level database opener that gets around anydbm/dbm
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
133 eccentricities.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
134 '''
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
135 # figure the class db type
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
136 path = os.path.join(os.getcwd(), self.dir, self.name)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
137 if self._db_type is None:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
138 self.cache_db_type(path)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
139
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
140 db_type = self._db_type
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
141
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
142 # new database? let anydbm pick the best dbm
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
143 if not db_type:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
144 return anydbm.open(path, 'c')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
145
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
146 # open the database with the correct module
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
147 dbm = __import__(db_type)
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
148
6036
5eeb073e33c7 flake8 whitespace fixes, remove unused exception var.
John Rouillard <rouilj@ieee.org>
parents: 5837
diff changeset
149 retries_left = 15
6565
2c2dbfc332ba Try to handle multiple connections better.
John Rouillard <rouilj@ieee.org>
parents: 6064
diff changeset
150 logger = logging.getLogger('roundup.hyperdb.backend.sessions')
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
151 while True:
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
152 try:
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
153 handle = dbm.open(path, mode)
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
154 break
6565
2c2dbfc332ba Try to handle multiple connections better.
John Rouillard <rouilj@ieee.org>
parents: 6064
diff changeset
155 except OSError as e:
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
156 # Primarily we want to catch and retry:
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
157 # [Errno 11] Resource temporarily unavailable retry
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
158 # FIXME: make this more specific
6565
2c2dbfc332ba Try to handle multiple connections better.
John Rouillard <rouilj@ieee.org>
parents: 6064
diff changeset
159 if retries_left < 10:
6680
b4d0b48b3096 Report partial file name when logging dbm.open failure/retry
John Rouillard <rouilj@ieee.org>
parents: 6565
diff changeset
160 logger.warning('dbm.open failed on ...%s, retry %s left: %s, %s'%(path[-15:],15-retries_left,retries_left,e))
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
161 if retries_left < 0:
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
162 # We have used up the retries. Reraise the exception
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
163 # that got us here.
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
164 raise
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
165 else:
6565
2c2dbfc332ba Try to handle multiple connections better.
John Rouillard <rouilj@ieee.org>
parents: 6064
diff changeset
166 # stagger retry to try to get around thundering herd issue.
2c2dbfc332ba Try to handle multiple connections better.
John Rouillard <rouilj@ieee.org>
parents: 6064
diff changeset
167 time.sleep(random.randint(0,25)*.005)
6064
bef1e42be04c Flake8 whitespace changes.
John Rouillard <rouilj@ieee.org>
parents: 6036
diff changeset
168 retries_left = retries_left - 1
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
169 continue # the while loop
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
170 return handle
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
171
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
172 def commit(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
173 pass
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
174
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
175 def lifetime(self, key_lifetime=0):
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
176 """Return the proper timestamp for a key with key_lifetime specified
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
177 in seconds. Default lifetime is 0.
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
178 """
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
179 now = time.time()
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
180 week = 60*60*24*7
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
181 return now - week + key_lifetime
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
182
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
183 def close(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
184 pass
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
185
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
186 def updateTimestamp(self, sessid):
3606
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
187 ''' don't update every hit - once a minute should be OK '''
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
188 sess = self.get(sessid, '__timestamp', None)
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
189 now = time.time()
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
190 if sess is None or now > sess + 60:
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
191 self.set(sessid, __timestamp=now)
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
192
3989
0112e9e1d068 improvements to session management
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
193 def clean(self):
0112e9e1d068 improvements to session management
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
194 ''' Remove session records that haven't been used for a week. '''
0112e9e1d068 improvements to session management
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
195 now = time.time()
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
196 week = 60*60*24*7
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
197 for sessid in self.list():
2151
ae21c1fc41d5 better fix
Richard Jones <richard@users.sourceforge.net>
parents: 2150
diff changeset
198 sess = self.get(sessid, '__timestamp', None)
ae21c1fc41d5 better fix
Richard Jones <richard@users.sourceforge.net>
parents: 2150
diff changeset
199 if sess is None:
ae21c1fc41d5 better fix
Richard Jones <richard@users.sourceforge.net>
parents: 2150
diff changeset
200 self.updateTimestamp(sessid)
3989
0112e9e1d068 improvements to session management
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
201 continue
2151
ae21c1fc41d5 better fix
Richard Jones <richard@users.sourceforge.net>
parents: 2150
diff changeset
202 interval = now - sess
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
203 if interval > week:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
204 self.destroy(sessid)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
205
6064
bef1e42be04c Flake8 whitespace changes.
John Rouillard <rouilj@ieee.org>
parents: 6036
diff changeset
206
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
207 class Sessions(BasicDatabase):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
208 name = 'sessions'
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
209
6064
bef1e42be04c Flake8 whitespace changes.
John Rouillard <rouilj@ieee.org>
parents: 6036
diff changeset
210
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
211 class OneTimeKeys(BasicDatabase):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
212 name = 'otks'
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
213
3925
603ec9630b08 i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3924
diff changeset
214 # vim: set sts ts=4 sw=4 et si :
Roundup Issue Tracker: http://roundup-tracker.org/