Mercurial > p > roundup > code

annotate roundup/backends/sessions_sqlite.py @ 6808:375d40a9e730

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add tests to BasicDatabase and merge implementations test/session_common.py: add new tests: * test set with bad __timestamps * test get on missing item with no default * test clear() method * test new lifetime() method everything below here was needed to get the tests to work across all db's. roundup/backends/sessions_dbm.py: make set() validate __timestamp as float. If invalid and item is new, set it to time.time(). If invalid and item exists keep original timestamp. add lifetime(key_lifetime) method. Given key_lifetime in seconds, generate a __timestamp that will be deleted after that many seconds. roundup/backends/sessions_rdbms.py: make set() behave the same as session_dbm. add lifetime method as above. roundup/backends/sessions_sqlite.py: import session_rdbms::BasicDatabase and override __init__. rather than cloning all the code. Kept a few logging and sql methods. roundup/test/memorydb.py: make get() on a missing key return KeyError make set() conform to dbm/rdbms implementations.
author John Rouillard <rouilj@ieee.org>
date Mon, 25 Jul 2022 21:21:26 -0400
parents 25d08e15e3b4
children fe0091279f50
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6804
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
1 """This module defines a very basic store that's used by the CGI interface
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
2 to store session and one-time-key information.
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
3
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
4 Yes, it's called "sessions" - because originally it only defined a session
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
5 class. It's now also used for One Time Key handling too.
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
6
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
7 We needed to split commits to session/OTK database from commits on the
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
8 main db structures (user data). This required two connections to the
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
9 sqlite db, which wasn't supported. This module was created so sqlite
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
10 didn't have to use dbm for the session/otk data. It hopefully will
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
11 provide a performance speedup.
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
12 """
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
13 __docformat__ = 'restructuredtext'
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
14 import os, time, logging
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
15
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
16 from roundup.anypy.html import html_escape as escape
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
17 import roundup.backends.sessions_rdbms as rdbms_session
6804
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
18
6808
375d40a9e730 Add tests to BasicDatabase and merge implementations
John Rouillard <rouilj@ieee.org>
parents: 6804
diff changeset
19 class BasicDatabase(rdbms_session.BasicDatabase):
6804
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
20 ''' Provide a nice encapsulation of an RDBMS table.
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
21
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
22 Keys are id strings, values are automatically marshalled data.
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
23 '''
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
24 name = None
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
25 def __init__(self, db):
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
26 self.db = db
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
27 self.conn, self.cursor = self.db.sql_open_connection(dbname=self.name)
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
28
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
29 self.sql('''SELECT name FROM sqlite_master WHERE type='table' AND name='%ss';'''%self.name)
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
30 table_exists = self.cursor.fetchone()
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
31
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
32 if not table_exists:
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
33 # create table/rows etc.
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
34 self.sql('''CREATE TABLE %(name)ss (%(name)s_key VARCHAR(255),
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
35 %(name)s_value TEXT, %(name)s_time REAL)'''%{"name":self.name})
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
36 self.sql('CREATE INDEX %(name)s_key_idx ON %(name)ss(%(name)s_key)'%{"name":self.name})
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
37 self.commit()
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
38
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
39 def log_debug(self, msg, *args, **kwargs):
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
40 """Log a message with level DEBUG."""
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
41
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
42 logger = self.get_logger()
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
43 logger.debug(msg, *args, **kwargs)
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
44
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
45 def log_info(self, msg, *args, **kwargs):
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
46 """Log a message with level INFO."""
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
47
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
48 logger = self.get_logger()
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
49 logger.info(msg, *args, **kwargs)
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
50
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
51 def get_logger(self):
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
52 """Return the logger for this database."""
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
53
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
54 # Because getting a logger requires acquiring a lock, we want
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
55 # to do it only once.
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
56 if not hasattr(self, '__logger'):
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
57 self.__logger = logging.getLogger('roundup')
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
58
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
59 return self.__logger
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
60
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
61 def sql(self, sql, args=None, cursor=None):
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
62 """ Execute the sql with the optional args.
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
63 """
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
64 self.log_debug('SQL %r %r' % (sql, args))
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
65 if not cursor:
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
66 cursor = self.cursor
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
67 if args:
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
68 cursor.execute(sql, args)
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
69 else:
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
70 cursor.execute(sql)
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
71
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
72 class Sessions(BasicDatabase):
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
73 name = 'session'
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
74
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
75 class OneTimeKeys(BasicDatabase):
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
76 name = 'otk'
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
77
25d08e15e3b4 issue2551224 - Replace dbm db for sessions/otks when using sqlite
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
78 # vim: set et sts=4 sw=4 :
Roundup Issue Tracker: http://roundup-tracker.org/