Mercurial > p > roundup > code

annotate roundup/backends/sessions_redis.py @ 7750:216662fbaaee

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
fix(i18n): fix incorrect lookup of some translations The code had: _("some term %s here" % term) this extracts the template, but looks up the string with %s replaced. So the translation is broken. Changed to: _("some term %s here") % term which looks up the template and substitutes in the translation of the template. Found by ruff INT ruleset.
author John Rouillard <rouilj@ieee.org>
date Fri, 01 Mar 2024 14:04:05 -0500
parents dd15c307c409
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
1 """This module defines a redis based store that's used by
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
2 the CGI interface to store session and one-time-key
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
3 information.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
4
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
5 Yes, it's called "sessions" - because originally it only
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
6 defined a session class. It's now also used for One Time Key
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
7 handling too.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
8
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
9 It uses simple strings rather than redis hash structure
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
10 because the hash the values are always strings. We need to
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
11 be able to represent the same data types available to rdbms
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
12 and dbm session stores.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
13
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
14 session_dbm uses marshal.dumps and marshal.loads. This seems
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
15 4 or 18 times faster than the repr()/eval() used by
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
16 session_rdbms. So use marshal even though it is impossible
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
17 to read when viewing (using redis-cli).
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
18 """
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
19 __docformat__ = 'restructuredtext'
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
20
7080
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
21 import marshal
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
22 import redis
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
23 import time
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
24
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
25 from roundup.anypy.html import html_escape as escape
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
26
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
27 from roundup.i18n import _
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
28
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6814
diff changeset
29 from roundup.backends.sessions_common import SessionCommon
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
30
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6814
diff changeset
31
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6814
diff changeset
32 class BasicDatabase(SessionCommon):
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
33 ''' Provide a nice encapsulation of a redis store.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
34
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
35 Keys are id strings, values are automatically marshalled data.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
36 '''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
37 name = None
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
38 default_lifetime = 60*60*24*7 # 1 week
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
39
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
40 # FIXME: figure out how to allow admin to change this
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
41 # to repr/eval using interfaces.py or other method.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
42 # marshalled data is not readable when debugging.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
43 tostr = marshal.dumps
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
44 todict = marshal.loads
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
45
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
46 def __init__(self, db):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
47 self.config = db.config
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
48 url = self.config.SESSIONDB_REDIS_URL
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
49
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
50 # Example at default port without auth.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
51 # redis://localhost:6379/0?health_check_interval=2
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
52 #
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
53 # Do not allow decode_responses=True in url, data is
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
54 # marshal'ed binary data that will get broken by decoding.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
55 # Enforce this in configuration.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
56 self.redis = redis.Redis.from_url(url=url, decode_responses=False)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
57
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
58 def makekey(self, key):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
59 '''method to namespace all keys using self.name:....'''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
60 return "%s:%s" % (self.name, key)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
61
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
62 def exists(self, infoid):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
63 return self.redis.exists(self.makekey(infoid))
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
64
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
65 def clear(self):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
66 '''Delete all keys from the database.'''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
67 self.redis.flushdb()
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
68
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
69 _marker = []
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
70
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
71 def get(self, infoid, value, default=_marker):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
72 '''get a specific value from the data associated with a key'''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
73 infoid = self.makekey(infoid)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
74 v = self.redis.get(infoid)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
75 if not v:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
76 if default != self._marker:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
77 return default
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
78 raise KeyError(_('Key %(key)s not found in %(name)s '
7750
216662fbaaee fix(i18n): fix incorrect lookup of some translations
John Rouillard <rouilj@ieee.org>
parents: 7080
diff changeset
79 'database.') % {"name": self.name,
216662fbaaee fix(i18n): fix incorrect lookup of some translations
John Rouillard <rouilj@ieee.org>
parents: 7080
diff changeset
80 "key": escape(infoid)})
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
81 return self.todict(v)[value]
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
82
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
83 def getall(self, infoid):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
84 '''return all values associated with a key'''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
85 try:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
86 d = self.redis.get(self.makekey(infoid))
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
87 if d is not None:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
88 d = self.todict(d)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
89 else:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
90 d = {}
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
91 del d['__timestamp']
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
92 return d
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
93 except KeyError:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
94 # It is possible for d to be malformed missing __timestamp.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
95 # If so, we get a misleading error, but anydbm does the
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
96 # same so....
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
97 raise KeyError(_('Key %(key)s not found in %(name)s '
7750
216662fbaaee fix(i18n): fix incorrect lookup of some translations
John Rouillard <rouilj@ieee.org>
parents: 7080
diff changeset
98 'database.') % {"name": self.name,
216662fbaaee fix(i18n): fix incorrect lookup of some translations
John Rouillard <rouilj@ieee.org>
parents: 7080
diff changeset
99 "key": escape(infoid)})
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
100
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
101 ''' def set_no_tranaction(self, infoid, **newvalues):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
102 """ this is missing transaction and may be affected by
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
103 a race condition on update. This will work for
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
104 redis-like embedded databases that don't support
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
105 watch/multi/exec
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
106 """
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
107 infoid = self.makekey(infoid)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
108 timestamp=None
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
109 values = self.redis.get(infoid)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
110 if values is not None:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
111 values = self.todict(values)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
112 else:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
113 values={}
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
114 try:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
115 timestamp = float(values['__timestamp'])
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
116 except KeyError:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
117 pass # stay at None
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
118
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
119 if '__timestamp' in newvalues:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
120 try:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
121 float(newvalues['__timestamp'])
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
122 except ValueError:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
123 # keep original timestamp if present
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
124 newvalues['__timestamp'] = timestamp or \
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
125 (time.time() + self.default_lifetime)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
126 else:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
127 newvalues['__timestamp'] = time.time() + self.default_lifetime
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
128
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
129 values.update(newvalues)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
130
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
131 self.redis.set(infoid, self.tostr(values))
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
132 self.redis.expireat(infoid, int(values['__timestamp']))
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
133 '''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
134
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
135 def set(self, infoid, **newvalues):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
136 """ Implement set using watch/multi/exec to get some
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
137 protection against a change committing between
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
138 getting the data and setting new fields and
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
139 saving.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
140 """
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
141 infoid = self.makekey(infoid)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
142 timestamp = None
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
143
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
144 with self.redis.pipeline() as transaction:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
145 # Give up and log after three tries.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
146 # Do not loop forever.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
147 for _retry in [1, 2, 3]:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
148 # I am ignoring transaction return values.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
149 # Assuming all errors will be via exceptions.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
150 # Not clear that return values that useful.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
151 transaction.watch(infoid)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
152 values = transaction.get(infoid)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
153 if values is not None:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
154 values = self.todict(values)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
155 else:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
156 values = {}
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
157
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
158 try:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
159 timestamp = float(values['__timestamp'])
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
160 except KeyError:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
161 pass # stay at None
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
162
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
163 if '__timestamp' in newvalues:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
164 try:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
165 float(newvalues['__timestamp'])
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
166 except ValueError:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
167 # keep original timestamp if present
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
168 newvalues['__timestamp'] = timestamp or \
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
169 (time.time() + self.default_lifetime)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
170 else:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
171 newvalues['__timestamp'] = time.time() + \
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
172 self.default_lifetime
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
173
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
174 values.update(newvalues)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
175
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
176 transaction.multi()
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
177 transaction.set(infoid, self.tostr(values))
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
178 transaction.expireat(infoid, int(values['__timestamp']))
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
179 try:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
180 # assume this works or raises an WatchError
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
181 # exception indicating I need to retry.
7080
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
182 # Since this is not a real transaction, an error
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
183 # in one step doesn't roll back other changes.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
184 # so I again ignore the return codes as it is not
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
185 # clear that I can do the rollback myself.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
186 # Format and other errors (e.g. expireat('d', 'd'))
7080
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
187 # raise exceptions that bubble up and result in mail
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
188 # to admin.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
189 transaction.execute()
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
190 break
7080
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
191 except redis.exceptions.WatchError:
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6814
diff changeset
192 self.log_info(
7750
216662fbaaee fix(i18n): fix incorrect lookup of some translations
John Rouillard <rouilj@ieee.org>
parents: 7080
diff changeset
193 _('Key %(key)s changed in %(name)s db') %
216662fbaaee fix(i18n): fix incorrect lookup of some translations
John Rouillard <rouilj@ieee.org>
parents: 7080
diff changeset
194 {"key": escape(infoid), "name": self.name}
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
195 )
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
196 else:
7080
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
197 try:
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
198 username = values['user']
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
199 except KeyError:
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
200 username = "Not Set"
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
201
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
202 raise Exception(
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
203 _("Redis set failed after %(retries)d retries for "
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
204 "user %(user)s with key %(key)s") % {
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
205 "key": escape(infoid), "retries": _retry,
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
206 "user": username})
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
207
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
208 def list(self):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
209 return list(self.redis.keys(self.makekey('*')))
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
210
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
211 def destroy(self, infoid=None):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
212 '''use unlink rather than delete as unlink is async and doesn't
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
213 wait for memory to be freed server-side
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
214 '''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
215 self.redis.unlink(self.makekey(infoid))
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
216
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
217 def commit(self):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
218 ''' no-op '''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
219 pass
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
220
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
221 def lifetime(self, key_lifetime=None):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
222 """Return the proper timestamp to expire a key with key_lifetime
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
223 specified in seconds. Default lifetime is self.default_lifetime.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
224 """
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
225 return time.time() + (key_lifetime or self.default_lifetime)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
226
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
227 def updateTimestamp(self, sessid):
7080
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
228 ''' Even Redis can be overwhelmed by multiple updates, so
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
229 only update if old session key is > 60 seconds old.
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
230 '''
7080
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
231 sess = self.get(sessid, '__timestamp', None)
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
232 now = time.time()
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
233 # unlike other session backends, __timestamp is not set to now
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
234 # but now + lifetime.
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
235 if sess is None or now > sess + 60 - self.default_lifetime:
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
236 lifetime = self.lifetime()
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
237 # note set also updates the expireat on the key in redis
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
238 self.set(sessid, __timestamp=lifetime)
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
239
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
240 def clean(self):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
241 ''' redis handles key expiration, so nothing to do here.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
242 '''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
243 pass
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
244
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
245 def close(self):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
246 ''' redis uses a connection pool that self manages, so nothing
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
247 to do on close.'''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
248 pass
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
249
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
250
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
251 class Sessions(BasicDatabase):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
252 name = 'sessions'
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
253
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
254
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
255 class OneTimeKeys(BasicDatabase):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
256 name = 'otks'
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
257
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
258 # vim: set sts ts=4 sw=4 et si :
Roundup Issue Tracker: http://roundup-tracker.org/