Mercurial > p > roundup > code

annotate roundup/backends/sessions_redis.py @ 7800:2d4684e4702d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
fix: enhancement to history command output and % template fix. Rather than using the key field, use the label field for descriptions. Call cls.labelprop(default_to_id=True) so it returns id rather than the first sorted property name. If labelprop() returns 'id' or 'title', we return nothing. 'id' means there is no label set and no properties named 'name' or 'title'. So have the caller do whatever it wants (prepend classname for example) when there is no human readable name. This prevents %(name)s%(key)s from producing: 23(23). Also don't accept the 'title' property. Titles can be too long. Arguably we could: '%(name)20s' to limit the title length. However without ellipses or something truncating the title might be confusing. So again pretend there is no human readable name.
author John Rouillard <rouilj@ieee.org>
date Tue, 12 Mar 2024 11:52:17 -0400
parents 216662fbaaee
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
1 """This module defines a redis based store that's used by
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
2 the CGI interface to store session and one-time-key
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
3 information.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
4
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
5 Yes, it's called "sessions" - because originally it only
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
6 defined a session class. It's now also used for One Time Key
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
7 handling too.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
8
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
9 It uses simple strings rather than redis hash structure
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
10 because the hash the values are always strings. We need to
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
11 be able to represent the same data types available to rdbms
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
12 and dbm session stores.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
13
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
14 session_dbm uses marshal.dumps and marshal.loads. This seems
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
15 4 or 18 times faster than the repr()/eval() used by
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
16 session_rdbms. So use marshal even though it is impossible
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
17 to read when viewing (using redis-cli).
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
18 """
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
19 __docformat__ = 'restructuredtext'
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
20
7080
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
21 import marshal
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
22 import redis
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
23 import time
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
24
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
25 from roundup.anypy.html import html_escape as escape
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
26
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
27 from roundup.i18n import _
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
28
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6814
diff changeset
29 from roundup.backends.sessions_common import SessionCommon
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
30
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6814
diff changeset
31
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6814
diff changeset
32 class BasicDatabase(SessionCommon):
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
33 ''' Provide a nice encapsulation of a redis store.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
34
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
35 Keys are id strings, values are automatically marshalled data.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
36 '''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
37 name = None
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
38 default_lifetime = 60*60*24*7 # 1 week
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
39
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
40 # FIXME: figure out how to allow admin to change this
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
41 # to repr/eval using interfaces.py or other method.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
42 # marshalled data is not readable when debugging.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
43 tostr = marshal.dumps
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
44 todict = marshal.loads
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
45
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
46 def __init__(self, db):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
47 self.config = db.config
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
48 url = self.config.SESSIONDB_REDIS_URL
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
49
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
50 # Example at default port without auth.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
51 # redis://localhost:6379/0?health_check_interval=2
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
52 #
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
53 # Do not allow decode_responses=True in url, data is
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
54 # marshal'ed binary data that will get broken by decoding.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
55 # Enforce this in configuration.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
56 self.redis = redis.Redis.from_url(url=url, decode_responses=False)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
57
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
58 def makekey(self, key):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
59 '''method to namespace all keys using self.name:....'''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
60 return "%s:%s" % (self.name, key)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
61
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
62 def exists(self, infoid):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
63 return self.redis.exists(self.makekey(infoid))
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
64
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
65 def clear(self):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
66 '''Delete all keys from the database.'''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
67 self.redis.flushdb()
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
68
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
69 _marker = []
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
70
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
71 def get(self, infoid, value, default=_marker):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
72 '''get a specific value from the data associated with a key'''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
73 infoid = self.makekey(infoid)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
74 v = self.redis.get(infoid)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
75 if not v:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
76 if default != self._marker:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
77 return default
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
78 raise KeyError(_('Key %(key)s not found in %(name)s '
7750
216662fbaaee fix(i18n): fix incorrect lookup of some translations
John Rouillard <rouilj@ieee.org>
parents: 7080
diff changeset
79 'database.') % {"name": self.name,
216662fbaaee fix(i18n): fix incorrect lookup of some translations
John Rouillard <rouilj@ieee.org>
parents: 7080
diff changeset
80 "key": escape(infoid)})
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
81 return self.todict(v)[value]
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
82
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
83 def getall(self, infoid):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
84 '''return all values associated with a key'''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
85 try:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
86 d = self.redis.get(self.makekey(infoid))
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
87 if d is not None:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
88 d = self.todict(d)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
89 else:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
90 d = {}
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
91 del d['__timestamp']
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
92 return d
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
93 except KeyError:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
94 # It is possible for d to be malformed missing __timestamp.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
95 # If so, we get a misleading error, but anydbm does the
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
96 # same so....
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
97 raise KeyError(_('Key %(key)s not found in %(name)s '
7750
216662fbaaee fix(i18n): fix incorrect lookup of some translations
John Rouillard <rouilj@ieee.org>
parents: 7080
diff changeset
98 'database.') % {"name": self.name,
216662fbaaee fix(i18n): fix incorrect lookup of some translations
John Rouillard <rouilj@ieee.org>
parents: 7080
diff changeset
99 "key": escape(infoid)})
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
100
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
101 ''' def set_no_tranaction(self, infoid, **newvalues):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
102 """ this is missing transaction and may be affected by
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
103 a race condition on update. This will work for
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
104 redis-like embedded databases that don't support
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
105 watch/multi/exec
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
106 """
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
107 infoid = self.makekey(infoid)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
108 timestamp=None
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
109 values = self.redis.get(infoid)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
110 if values is not None:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
111 values = self.todict(values)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
112 else:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
113 values={}
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
114 try:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
115 timestamp = float(values['__timestamp'])
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
116 except KeyError:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
117 pass # stay at None
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
118
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
119 if '__timestamp' in newvalues:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
120 try:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
121 float(newvalues['__timestamp'])
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
122 except ValueError:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
123 # keep original timestamp if present
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
124 newvalues['__timestamp'] = timestamp or \
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
125 (time.time() + self.default_lifetime)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
126 else:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
127 newvalues['__timestamp'] = time.time() + self.default_lifetime
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
128
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
129 values.update(newvalues)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
130
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
131 self.redis.set(infoid, self.tostr(values))
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
132 self.redis.expireat(infoid, int(values['__timestamp']))
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
133 '''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
134
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
135 def set(self, infoid, **newvalues):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
136 """ Implement set using watch/multi/exec to get some
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
137 protection against a change committing between
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
138 getting the data and setting new fields and
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
139 saving.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
140 """
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
141 infoid = self.makekey(infoid)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
142 timestamp = None
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
143
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
144 with self.redis.pipeline() as transaction:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
145 # Give up and log after three tries.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
146 # Do not loop forever.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
147 for _retry in [1, 2, 3]:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
148 # I am ignoring transaction return values.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
149 # Assuming all errors will be via exceptions.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
150 # Not clear that return values that useful.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
151 transaction.watch(infoid)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
152 values = transaction.get(infoid)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
153 if values is not None:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
154 values = self.todict(values)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
155 else:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
156 values = {}
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
157
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
158 try:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
159 timestamp = float(values['__timestamp'])
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
160 except KeyError:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
161 pass # stay at None
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
162
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
163 if '__timestamp' in newvalues:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
164 try:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
165 float(newvalues['__timestamp'])
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
166 except ValueError:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
167 # keep original timestamp if present
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
168 newvalues['__timestamp'] = timestamp or \
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
169 (time.time() + self.default_lifetime)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
170 else:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
171 newvalues['__timestamp'] = time.time() + \
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
172 self.default_lifetime
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
173
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
174 values.update(newvalues)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
175
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
176 transaction.multi()
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
177 transaction.set(infoid, self.tostr(values))
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
178 transaction.expireat(infoid, int(values['__timestamp']))
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
179 try:
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
180 # assume this works or raises an WatchError
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
181 # exception indicating I need to retry.
7080
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
182 # Since this is not a real transaction, an error
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
183 # in one step doesn't roll back other changes.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
184 # so I again ignore the return codes as it is not
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
185 # clear that I can do the rollback myself.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
186 # Format and other errors (e.g. expireat('d', 'd'))
7080
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
187 # raise exceptions that bubble up and result in mail
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
188 # to admin.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
189 transaction.execute()
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
190 break
7080
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
191 except redis.exceptions.WatchError:
6823
fe0091279f50 Refactor session db logging and key generation for sessions/otks
John Rouillard <rouilj@ieee.org>
parents: 6814
diff changeset
192 self.log_info(
7750
216662fbaaee fix(i18n): fix incorrect lookup of some translations
John Rouillard <rouilj@ieee.org>
parents: 7080
diff changeset
193 _('Key %(key)s changed in %(name)s db') %
216662fbaaee fix(i18n): fix incorrect lookup of some translations
John Rouillard <rouilj@ieee.org>
parents: 7080
diff changeset
194 {"key": escape(infoid), "name": self.name}
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
195 )
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
196 else:
7080
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
197 try:
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
198 username = values['user']
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
199 except KeyError:
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
200 username = "Not Set"
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
201
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
202 raise Exception(
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
203 _("Redis set failed after %(retries)d retries for "
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
204 "user %(user)s with key %(key)s") % {
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
205 "key": escape(infoid), "retries": _retry,
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
206 "user": username})
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
207
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
208 def list(self):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
209 return list(self.redis.keys(self.makekey('*')))
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
210
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
211 def destroy(self, infoid=None):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
212 '''use unlink rather than delete as unlink is async and doesn't
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
213 wait for memory to be freed server-side
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
214 '''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
215 self.redis.unlink(self.makekey(infoid))
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
216
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
217 def commit(self):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
218 ''' no-op '''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
219 pass
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
220
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
221 def lifetime(self, key_lifetime=None):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
222 """Return the proper timestamp to expire a key with key_lifetime
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
223 specified in seconds. Default lifetime is self.default_lifetime.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
224 """
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
225 return time.time() + (key_lifetime or self.default_lifetime)
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
226
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
227 def updateTimestamp(self, sessid):
7080
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
228 ''' Even Redis can be overwhelmed by multiple updates, so
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
229 only update if old session key is > 60 seconds old.
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
230 '''
7080
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
231 sess = self.get(sessid, '__timestamp', None)
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
232 now = time.time()
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
233 # unlike other session backends, __timestamp is not set to now
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
234 # but now + lifetime.
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
235 if sess is None or now > sess + 60 - self.default_lifetime:
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
236 lifetime = self.lifetime()
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
237 # note set also updates the expireat on the key in redis
dd15c307c409 bug: fix undefined symbol in exception; limit session update time
John Rouillard <rouilj@ieee.org>
parents: 6823
diff changeset
238 self.set(sessid, __timestamp=lifetime)
6814
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
239
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
240 def clean(self):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
241 ''' redis handles key expiration, so nothing to do here.
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
242 '''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
243 pass
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
244
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
245 def close(self):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
246 ''' redis uses a connection pool that self manages, so nothing
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
247 to do on close.'''
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
248 pass
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
249
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
250
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
251 class Sessions(BasicDatabase):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
252 name = 'sessions'
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
253
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
254
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
255 class OneTimeKeys(BasicDatabase):
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
256 name = 'otks'
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
257
3f60a71b0812 Summary: Support selecion session/otk data store. Add redis as data store.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
258 # vim: set sts ts=4 sw=4 et si :
Roundup Issue Tracker: http://roundup-tracker.org/