Mercurial > p > roundup > code

annotate roundup/backends/sessions_dbm.py @ 6823:fe0091279f50

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Refactor session db logging and key generation for sessions/otks While I was working on the redis sessiondb stuff, I noticed that log_wanrning, get_logger ... was duplicated. Also there was code to generate a unique key for otks that was duplicated. Changes: creating new sessions_common.py and SessionsCommon class to provide methods: log_warning, log_info, log_debug, get_logger, getUniqueKey getUniqueKey method is closer to the method used to make session keys in client.py. sessions_common.py now report when random_.py chooses a weak random number generator. Removed same from rest.py. get_logger reconciles all logging under roundup.hyperdb.backends.<name of BasicDatabase class> some backends used to log to root logger. have BasicDatabase in other sessions_*.py modules inherit from SessionCommon. change logging to use log_* methods. In addition: remove unused imports reported by flake8 and other formatting changes modify actions.py, rest.py, templating.py to use getUniqueKey method. add tests for new methods test_redis_session.py swap out ModuleNotFoundError for ImportError to prevent crash in python2 when redis is not present. allow injection of username:password or just password into redis connection URL. set pytest_redis_pw envirnment variable to password or user:password when running test.
author John Rouillard <rouilj@ieee.org>
date Sun, 07 Aug 2022 01:51:11 -0400
parents 375d40a9e730
children 39c482e6a246
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1 """This module defines a very basic store that's used by the CGI interface
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2 to store session and one-time-key information.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
4 Yes, it's called "sessions" - because originally it only defined a session
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
5 class. It's now also used for One Time Key handling too.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
6 """
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
7 __docformat__ = 'restructuredtext'
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
8
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
9 import marshal, os, random, time
4362
74476eaac38a more modernisation
Richard Jones <richard@users.sourceforge.net>
parents: 3989
diff changeset
10
5837
883c9e90b403 Fix problem with cgi.escape being depricated a different way. This way
John Rouillard <rouilj@ieee.org>
parents: 5806
diff changeset
11 from roundup.anypy.html import html_escape as escape
5806
abee2c2c822e More cgi.escape/html.escape fixes.
John Rouillard <rouilj@ieee.org>
parents: 5732
diff changeset
12
3924
21d3d7eeea8c assorted pyflakes fixes
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3609
diff changeset
13 from roundup import hyperdb
3925
603ec9630b08 i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3924
diff changeset
14 from roundup.i18n import _
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
15 from roundup.anypy.dbm_ import anydbm, whichdb
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
16 from roundup.backends.sessions_common import SessionCommon
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
17
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
18
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
19 class BasicDatabase(SessionCommon):
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
20 ''' Provide a nice encapsulation of an anydbm store.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
21
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
22 Keys are id strings, values are automatically marshalled data.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
23 '''
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
24 _db_type = None
5252
0b154486ed38 Make sure that the name property is initialized to none.
John Rouillard <rouilj@ieee.org>
parents: 5011
diff changeset
25 name = None
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
26
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
27 def __init__(self, db):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
28 self.config = db.config
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
29 self.dir = db.config.DATABASE
3609
f2fda3e6fc8b umask is now configurable (with the same 0002 default)
Richard Jones <richard@users.sourceforge.net>
parents: 3606
diff changeset
30 os.umask(db.config.UMASK)
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
31
2169
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
32 def exists(self, infoid):
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
33 db = self.opendb('c')
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
34 try:
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
35 return infoid in db
2169
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
36 finally:
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
37 db.close()
12cd4fa91eb7 OTK generation was busted (thanks Stuart D. Gathman)
Richard Jones <richard@users.sourceforge.net>
parents: 2151
diff changeset
38
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
39 def clear(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
40 path = os.path.join(self.dir, self.name)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
41 if os.path.exists(path):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
42 os.remove(path)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
43 elif os.path.exists(path+'.db'): # dbm appends .db
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
44 os.remove(path+'.db')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
45
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
46 def cache_db_type(self, path):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
47 ''' determine which DB wrote the class file, and cache it as an
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
48 attribute of __class__ (to allow for subclassed DBs to be
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
49 different sorts)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
50 '''
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
51 db_type = ''
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
52 if os.path.exists(path):
4363
Richard Jones <richard@users.sourceforge.net>
parents: 4362
diff changeset
53 db_type = whichdb(path)
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
54 if not db_type:
4362
74476eaac38a more modernisation
Richard Jones <richard@users.sourceforge.net>
parents: 3989
diff changeset
55 raise hyperdb.DatabaseError(
74476eaac38a more modernisation
Richard Jones <richard@users.sourceforge.net>
parents: 3989
diff changeset
56 _("Couldn't identify database type"))
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
57 elif os.path.exists(path+'.db'):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
58 # if the path ends in '.db', it's a dbm database, whether
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
59 # anydbm says it's dbhash or not!
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
60 db_type = 'dbm'
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
61 self.__class__._db_type = db_type
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
62
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
63 _marker = []
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
64
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
65 def get(self, infoid, value, default=_marker):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
66 db = self.opendb('c')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
67 try:
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
68 if infoid in db:
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
69 values = marshal.loads(db[infoid])
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
70 else:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
71 if default != self._marker:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
72 return default
6036
5eeb073e33c7 flake8 whitespace fixes, remove unused exception var.
John Rouillard <rouilj@ieee.org>
parents: 5837
diff changeset
73 raise KeyError('No such %s "%s"' % (self.name, escape(infoid)))
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
74 return values.get(value, None)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
75 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
76 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
77
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
78 def getall(self, infoid):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
79 db = self.opendb('c')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
80 try:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
81 try:
2089
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
82 d = marshal.loads(db[infoid])
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
83 del d['__timestamp']
93f03c6714d8 A few big changes in this commit:
Richard Jones <richard@users.sourceforge.net>
parents: 2082
diff changeset
84 return d
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
85 except KeyError:
6036
5eeb073e33c7 flake8 whitespace fixes, remove unused exception var.
John Rouillard <rouilj@ieee.org>
parents: 5837
diff changeset
86 raise KeyError('No such %s "%s"' % (self.name, escape(infoid)))
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
87 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
88 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
89
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
90 def set(self, infoid, **newvalues):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
91 db = self.opendb('c')
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
92 timestamp = None
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
93 try:
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
94 if infoid in db:
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
95 values = marshal.loads(db[infoid])
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
96 try:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
97 timestamp = values['__timestamp']
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
98 except KeyError:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
99 pass # stay at None
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
100 else:
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
101 values = {}
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
102
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
103 if '__timestamp' in newvalues:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
104 try:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
105 float(newvalues['__timestamp'])
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
106 except ValueError:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
107 # keep original timestamp if present
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
108 newvalues['__timestamp'] = timestamp or time.time()
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
109 else:
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
110 newvalues['__timestamp'] = time.time()
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
111
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
112 values.update(newvalues)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
113 db[infoid] = marshal.dumps(values)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
114 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
115 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
116
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
117 def list(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
118 db = self.opendb('r')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
119 try:
4385
8d135c19523e fix up some pre-Python2.6 compatibility issues in the *dbm interface
Richard Jones <richard@users.sourceforge.net>
parents: 4383
diff changeset
120 return list(db.keys())
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
121 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
122 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
123
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
124 def destroy(self, infoid):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
125 db = self.opendb('c')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
126 try:
5011
d5da643b3d25 Remove key_in() from roundup.anypy.dbm_
John Kristensen <john@jerrykan.com>
parents: 4585
diff changeset
127 if infoid in db:
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
128 del db[infoid]
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
129 finally:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
130 db.close()
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
131
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
132 def opendb(self, mode):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
133 '''Low-level database opener that gets around anydbm/dbm
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
134 eccentricities.
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
135 '''
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
136 # figure the class db type
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
137 path = os.path.join(os.getcwd(), self.dir, self.name)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
138 if self._db_type is None:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
139 self.cache_db_type(path)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
140
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
141 db_type = self._db_type
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
142
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
143 # new database? let anydbm pick the best dbm
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
144 if not db_type:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
145 return anydbm.open(path, 'c')
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
146
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
147 # open the database with the correct module
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
148 dbm = __import__(db_type)
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
149
6036
5eeb073e33c7 flake8 whitespace fixes, remove unused exception var.
John Rouillard <rouilj@ieee.org>
parents: 5837
diff changeset
150 retries_left = 15
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
151 while True:
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
152 try:
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
153 handle = dbm.open(path, mode)
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
154 break
6565
2c2dbfc332ba Try to handle multiple connections better.
John Rouillard <rouilj@ieee.org>
parents: 6064
diff changeset
155 except OSError as e:
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
156 # Primarily we want to catch and retry:
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
157 # [Errno 11] Resource temporarily unavailable retry
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
158 # FIXME: make this more specific
6565
2c2dbfc332ba Try to handle multiple connections better.
John Rouillard <rouilj@ieee.org>
parents: 6064
diff changeset
159 if retries_left < 10:
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
160 self.log_warning(
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
161 'dbm.open failed on ...%s, retry %s left: %s, %s' %
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
162 (path[-15:], 15-retries_left, retries_left, e))
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
163 if retries_left < 0:
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
164 # We have used up the retries. Reraise the exception
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
165 # that got us here.
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
166 raise
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
167 else:
6565
2c2dbfc332ba Try to handle multiple connections better.
John Rouillard <rouilj@ieee.org>
parents: 6064
diff changeset
168 # stagger retry to try to get around thundering herd issue.
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6808
diff changeset
169 time.sleep(random.randint(0, 25)*.005)
6064
bef1e42be04c Flake8 whitespace changes.
John Rouillard <rouilj@ieee.org>
parents: 6036
diff changeset
170 retries_left = retries_left - 1
5732
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
171 continue # the while loop
0e6ed3d72f92 Rest rate limiting code first commit. It is a bit rough and turned off
John Rouillard <rouilj@ieee.org>
parents: 5252
diff changeset
172 return handle
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
173
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
174 def commit(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
175 pass
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
176
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
177 def lifetime(self, key_lifetime=0):
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
178 """Return the proper timestamp for a key with key_lifetime specified
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
179 in seconds. Default lifetime is 0.
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
180 """
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
181 now = time.time()
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
182 week = 60*60*24*7
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
183 return now - week + key_lifetime
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6680
diff changeset
184
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
185 def close(self):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
186 pass
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
187
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
188 def updateTimestamp(self, sessid):
3606
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
189 ''' don't update every hit - once a minute should be OK '''
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
190 sess = self.get(sessid, '__timestamp', None)
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
191 now = time.time()
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
192 if sess is None or now > sess + 60:
04dc3eef67b7 reduced frequency of session timestamp update
Richard Jones <richard@users.sourceforge.net>
parents: 2169
diff changeset
193 self.set(sessid, __timestamp=now)
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
194
3989
0112e9e1d068 improvements to session management
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
195 def clean(self):
0112e9e1d068 improvements to session management
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
196 ''' Remove session records that haven't been used for a week. '''
0112e9e1d068 improvements to session management
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
197 now = time.time()
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
198 week = 60*60*24*7
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
199 for sessid in self.list():
2151
ae21c1fc41d5 better fix
Richard Jones <richard@users.sourceforge.net>
parents: 2150
diff changeset
200 sess = self.get(sessid, '__timestamp', None)
ae21c1fc41d5 better fix
Richard Jones <richard@users.sourceforge.net>
parents: 2150
diff changeset
201 if sess is None:
ae21c1fc41d5 better fix
Richard Jones <richard@users.sourceforge.net>
parents: 2150
diff changeset
202 self.updateTimestamp(sessid)
3989
0112e9e1d068 improvements to session management
Richard Jones <richard@users.sourceforge.net>
parents: 3925
diff changeset
203 continue
2151
ae21c1fc41d5 better fix
Richard Jones <richard@users.sourceforge.net>
parents: 2150
diff changeset
204 interval = now - sess
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
205 if interval > week:
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
206 self.destroy(sessid)
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
207
6064
bef1e42be04c Flake8 whitespace changes.
John Rouillard <rouilj@ieee.org>
parents: 6036
diff changeset
208
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
209 class Sessions(BasicDatabase):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
210 name = 'sessions'
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
211
6064
bef1e42be04c Flake8 whitespace changes.
John Rouillard <rouilj@ieee.org>
parents: 6036
diff changeset
212
2082
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
213 class OneTimeKeys(BasicDatabase):
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
214 name = 'otks'
c091cacdc505 Finished implementation of session and one-time-key stores for RDBMS backends.
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
215
3925
603ec9630b08 i18n for hyperdb and backend errors
Justus Pendleton <jpend@users.sourceforge.net>
parents: 3924
diff changeset
216 # vim: set sts ts=4 sw=4 et si :
Roundup Issue Tracker: http://roundup-tracker.org/