Mercurial > p > roundup > code

annotate roundup/backends/sessions_dbm.py @ 8543:1ffa1f42e1da

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
refactor: rework mime type comparison and clean code rest.py: accept application/* as match for application/json in non /binary_context rest path. allow defining default mime type to return when file/message is missing mime type. Make it a class variable to it can be changed from text/plain to text/markdown or whatever. extract code from determine_output_format() to create create_valid_content_types() method which returns a list of matching mime types for a given type/subtype. Eliminate mostly duplicate return statements by introducing a variable to specify valid mime types in error message. rest_common.py: Fix error messages that now return application/* as valid mime type. CHANGES.txt upgrading.txt rest.txt: top level notes and corrections. Also correct rst syntax on earlier change.
author John Rouillard <rouilj@ieee.org>
date Tue, 24 Mar 2026 21:30:47 -0400
parents e331be9bc473
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1 """This module defines a very basic store that's used by the CGI interface
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2 to store session and one-time-key information.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
4 Yes, it's called "sessions" - because originally it only defined a session
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
5 class. It's now also used for One Time Key handling too.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
6 """
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
7 __docformat__ = 'restructuredtext'
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
8
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
9 import marshal, os, random, time
4362
74476eaac38a more modernisation
Richard Jones <richard@users.sourceforge.net>
parents: 3989
diff changeset
10
5837
883c9e90b403 Fix problem with cgi.escape being depricated a different way. This way
John Rouillard <rouilj@ieee.org>
parents: 5806
diff changeset
11 from roundup.anypy.html import html_escape as escape
5806
abee2c2c822e More cgi.escape/html.escape fixes.
John Rouillard <rouilj@ieee.org>
parents: 5732
diff changeset
12
3924
21d3d7eeea8c assorted pyflakes fixes
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3609
diff changeset
13 from roundup import hyperdb
3925
603ec9630b08 i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3924
diff changeset
14 from roundup.i18n import _
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
15 from roundup.anypy.dbm_ import anydbm, whichdb
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
16 from roundup.backends.sessions_common import SessionCommon
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
17
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
18
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
19 class BasicDatabase(SessionCommon):
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
20 ''' Provide a nice encapsulation of an anydbm store.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
21
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
22 Keys are id strings, values are automatically marshalled data.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
23 '''
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
24 _db_type = None
5252
0b154486ed38 Make sure that the name property is initialized to none.
John Rouillard <rouilj@ieee.org>
parents: 5011
diff changeset
25 name = None
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
26
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
27 def __init__(self, db):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
28 self.config = db.config
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
29 self.dir = db.config.DATABASE
3609
f2fda3e6fc8b umask is now configurable (with the same 0002 default)
Richard Jones <richard@users.sourceforge.net>
parents: 3606
diff changeset
30 os.umask(db.config.UMASK)
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
31
2169
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
32 def exists(self, infoid):
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
33 db = self.opendb('c')
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
34 try:
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
35 return infoid in db
2169
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
36 finally:
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
37 db.close()
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
38
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
39 def clear(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
40 path = os.path.join(self.dir, self.name)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
41 if os.path.exists(path):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
42 os.remove(path)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
43 elif os.path.exists(path+'.db'): # dbm appends .db
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
44 os.remove(path+'.db')
7879
39c482e6a246 fix: fix code to make tests of session and otks databases pass on windows
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
45 elif os.path.exists(path+".dir"): # dumb dbm
39c482e6a246 fix: fix code to make tests of session and otks databases pass on windows
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
46 os.remove(path+".dir")
39c482e6a246 fix: fix code to make tests of session and otks databases pass on windows
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
47 os.remove(path+".dat")
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
48
7889
a5d39e5a5cdc fix: revert setting _db_type when creating new db.
John Rouillard <rouilj@ieee.org>
parents: 7888
diff changeset
49 #self._db_type = None
7888
456edf872259 fix: set _db_type to None when clearing anydbm db.
John Rouillard <rouilj@ieee.org>
parents: 7879
diff changeset
50
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
51 def cache_db_type(self, path):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
52 ''' determine which DB wrote the class file, and cache it as an
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
53 attribute of __class__ (to allow for subclassed DBs to be
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
54 different sorts)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
55 '''
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
56 db_type = ''
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
57 if os.path.exists(path):
4363
Richard Jones <richard@users.sourceforge.net>
parents: 4362
diff changeset
58 db_type = whichdb(path)
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
59 if not db_type:
4362
74476eaac38a more modernisation
Richard Jones <richard@users.sourceforge.net>
parents: 3989
diff changeset
60 raise hyperdb.DatabaseError(
74476eaac38a more modernisation
Richard Jones <richard@users.sourceforge.net>
parents: 3989
diff changeset
61 _("Couldn't identify database type"))
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
62 elif os.path.exists(path+'.db'):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
63 # if the path ends in '.db', it's a dbm database, whether
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
64 # anydbm says it's dbhash or not!
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
65 db_type = 'dbm'
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
66 self.__class__._db_type = db_type
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
67
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
68 _marker = []
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
69
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
70 def get(self, infoid, value, default=_marker):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
71 db = self.opendb('c')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
72 try:
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
73 if infoid in db:
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
74 values = marshal.loads(db[infoid])
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
75 else:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
76 if default != self._marker:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
77 return default
6036
5eeb073e33c7 flake8 whitespace fixes, remove unused exception var.
John Rouillard <rouilj@ieee.org>
parents: 5837
diff changeset
78 raise KeyError('No such %s "%s"' % (self.name, escape(infoid)))
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
79 return values.get(value, None)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
80 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
81 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
82
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
83 def getall(self, infoid):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
84 db = self.opendb('c')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
85 try:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
86 try:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
87 d = marshal.loads(db[infoid])
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
88 del d['__timestamp']
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
89 return d
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
90 except KeyError:
6036
5eeb073e33c7 flake8 whitespace fixes, remove unused exception var.
John Rouillard <rouilj@ieee.org>
parents: 5837
diff changeset
91 raise KeyError('No such %s "%s"' % (self.name, escape(infoid)))
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
92 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
93 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
94
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
95 def set(self, infoid, **newvalues):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
96 db = self.opendb('c')
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
97 timestamp = None
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
98 try:
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
99 if infoid in db:
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
100 values = marshal.loads(db[infoid])
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
101 try:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
102 timestamp = values['__timestamp']
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
103 except KeyError:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
104 pass # stay at None
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
105 else:
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
106 values = {}
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
107
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
108 if '__timestamp' in newvalues:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
109 try:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
110 float(newvalues['__timestamp'])
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
111 except ValueError:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
112 # keep original timestamp if present
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
113 newvalues['__timestamp'] = timestamp or time.time()
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
114 else:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
115 newvalues['__timestamp'] = time.time()
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
116
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
117 values.update(newvalues)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
118 db[infoid] = marshal.dumps(values)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
119 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
120 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
121
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
122 def list(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
123 db = self.opendb('r')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
124 try:
4385
8d135c19523e fix up some pre-Python2.6 compatibility issues in the *dbm interface
Richard Jones <richard@users.sourceforge.net>
parents: 4383
diff changeset
125 return list(db.keys())
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
126 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
127 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
128
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
129 def destroy(self, infoid):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
130 db = self.opendb('c')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
131 try:
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
132 if infoid in db:
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
133 del db[infoid]
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
134 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
135 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
136
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
137 def opendb(self, mode):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
138 '''Low-level database opener that gets around anydbm/dbm
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
139 eccentricities.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
140 '''
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
141 # figure the class db type
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
142 path = os.path.join(os.getcwd(), self.dir, self.name)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
143 if self._db_type is None:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
144 self.cache_db_type(path)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
145
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
146 db_type = self._db_type
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
147
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
148 # new database? let anydbm pick the best dbm
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
149 if not db_type:
7879
39c482e6a246 fix: fix code to make tests of session and otks databases pass on windows
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
150 db = anydbm.open(path, 'c')
7889
a5d39e5a5cdc fix: revert setting _db_type when creating new db.
John Rouillard <rouilj@ieee.org>
parents: 7888
diff changeset
151 #self.cache_db_type(path)
7879
39c482e6a246 fix: fix code to make tests of session and otks databases pass on windows
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
152 return db
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
153
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
154 # open the database with the correct module
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
155 dbm = __import__(db_type)
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
156
6036
5eeb073e33c7 flake8 whitespace fixes, remove unused exception var.
John Rouillard <rouilj@ieee.org>
parents: 5837
diff changeset
157 retries_left = 15
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
158 while True:
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
159 try:
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
160 handle = dbm.open(path, mode)
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
161 break
6565
2c2dbfc332ba Try to handle multiple connections better.
John Rouillard <rouilj@ieee.org>
parents: 6064
diff changeset
162 except OSError as e:
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
163 # Primarily we want to catch and retry:
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
164 # [Errno 11] Resource temporarily unavailable retry
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
165 # FIXME: make this more specific
6565
2c2dbfc332ba Try to handle multiple connections better.
John Rouillard <rouilj@ieee.org>
parents: 6064
diff changeset
166 if retries_left < 10:
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
167 self.log_warning(
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
168 'dbm.open failed on ...%s, retry %s left: %s, %s' %
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
169 (path[-15:], 15-retries_left, retries_left, e))
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
170 if retries_left < 0:
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
171 # We have used up the retries. Reraise the exception
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
172 # that got us here.
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
173 raise
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
174 else:
6565
2c2dbfc332ba Try to handle multiple connections better.
John Rouillard <rouilj@ieee.org>
parents: 6064
diff changeset
175 # stagger retry to try to get around thundering herd issue.
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
176 time.sleep(random.randint(0, 25)*.005)
6064
bef1e42be04c Flake8 whitespace changes.
John Rouillard <rouilj@ieee.org>
parents: 6036
diff changeset
177 retries_left = retries_left - 1
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
178 continue # the while loop
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
179 return handle
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
180
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
181 def commit(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
182 pass
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
183
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
184 def lifetime(self, key_lifetime=0):
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
185 """Return the proper timestamp for a key with key_lifetime specified
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
186 in seconds. Default lifetime is 0.
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
187 """
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
188 now = time.time()
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
189 week = 60*60*24*7
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
190 return now - week + key_lifetime
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
191
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
192 def close(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
193 pass
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
194
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
195 def updateTimestamp(self, sessid):
3606
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
196 ''' don't update every hit - once a minute should be OK '''
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
197 sess = self.get(sessid, '__timestamp', None)
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
198 now = time.time()
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
199 if sess is None or now > sess + 60:
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
200 self.set(sessid, __timestamp=now)
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
201
3989
0112e9e1d068 improvements to session management
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
202 def clean(self):
0112e9e1d068 improvements to session management
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
203 ''' Remove session records that haven't been used for a week. '''
8504
e331be9bc473 doc: add note that we can't incrementally clean old records from db
John Rouillard <rouilj@ieee.org>
parents: 8404
diff changeset
204 ''' Note: deletion of old keys must be completed when this method
e331be9bc473 doc: add note that we can't incrementally clean old records from db
John Rouillard <rouilj@ieee.org>
parents: 8404
diff changeset
205 returns. Calling code must not have any expired keys present
e331be9bc473 doc: add note that we can't incrementally clean old records from db
John Rouillard <rouilj@ieee.org>
parents: 8404
diff changeset
206 after this returns or expired keys could be used to validate
e331be9bc473 doc: add note that we can't incrementally clean old records from db
John Rouillard <rouilj@ieee.org>
parents: 8404
diff changeset
207 a user. This can mean a long delay when expiring but ....'''
3989
0112e9e1d068 improvements to session management
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
208 now = time.time()
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
209 week = 60*60*24*7
8404
accddaeda449 fix: perf improvement for session clean() and warning added
John Rouillard <rouilj@ieee.org>
parents: 7889
diff changeset
210 a_week_ago = now - week
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
211 for sessid in self.list():
2151
ae21c1fc41d5 better fix
Richard Jones <richard@users.sourceforge.net>
parents: 2150
diff changeset
212 sess = self.get(sessid, '__timestamp', None)
ae21c1fc41d5 better fix
Richard Jones <richard@users.sourceforge.net>
parents: 2150
diff changeset
213 if sess is None:
ae21c1fc41d5 better fix
Richard Jones <richard@users.sourceforge.net>
parents: 2150
diff changeset
214 self.updateTimestamp(sessid)
3989
0112e9e1d068 improvements to session management
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
215 continue
8404
accddaeda449 fix: perf improvement for session clean() and warning added
John Rouillard <rouilj@ieee.org>
parents: 7889
diff changeset
216 if a_week_ago > sess:
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
217 self.destroy(sessid)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
218
8404
accddaeda449 fix: perf improvement for session clean() and warning added
John Rouillard <rouilj@ieee.org>
parents: 7889
diff changeset
219 run_time = time.time() - now
accddaeda449 fix: perf improvement for session clean() and warning added
John Rouillard <rouilj@ieee.org>
parents: 7889
diff changeset
220 if run_time > 3:
accddaeda449 fix: perf improvement for session clean() and warning added
John Rouillard <rouilj@ieee.org>
parents: 7889
diff changeset
221 self.log_warning("clean() took %.2fs", run_time)
6064
bef1e42be04c Flake8 whitespace changes.
John Rouillard <rouilj@ieee.org>
parents: 6036
diff changeset
222
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
223 class Sessions(BasicDatabase):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
224 name = 'sessions'
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
225
6064
bef1e42be04c Flake8 whitespace changes.
John Rouillard <rouilj@ieee.org>
parents: 6036
diff changeset
226
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
227 class OneTimeKeys(BasicDatabase):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
228 name = 'otks'
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
229
3925
603ec9630b08 i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3924
diff changeset
230 # vim: set sts ts=4 sw=4 et si :
Roundup Issue Tracker: http://roundup-tracker.org/