Mercurial > p > roundup > code
comparison roundup/backends/back_sqlite.py @ 6814:3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
Allow admin to select the backend data store. Compatibility matrix:
main\/ session>| anydbm | sqlite | redis | mysql | postgresql |
anydbm | D | | X | | |
sqlite | X | D | X | | |
mysql | | | | D | |
postgresql | | | | | D |
--------------------------------------------------------------+
D - default if unconfigured, X - compatible choice
DETAILS
roundup/configuration.py:
add config.ini section sessiondb with settings: backend and redis_url.
CHANGES.txt, doc/admin_guide.txt, doc/installation.txt, doc/upgrading.txt:
doc on config of session db and redis. Plus some other fixes:
admin - clarified why we do not drop __words and __testids
table in native-fts conversion. TYpo fix.
upgrading - doc how you can keep using anydbm for session data with
sqlite. Fix dupe sentence in an upgrading config.ini
section.
roundup/backends/back_anydbm.py, roundup/backends/back_sqlite.py:
code to support redis, redis/anydbm backends respectively.
roundup/backends/sessions_redis.py
new storage backend for redis.
roundup/rest.py, roundup/cgi/actions.py, roundup/cgi/templating.py
redis uses a different way of calculating lifetime/timestamp.
Since expiration of an item occurred if its timestamp was more
than 1 week old, code would calculate:
now - 1 week + lifetime.
But this results in faster expiration in redis if used for
lifetime/timestamp.
Convert code to use the lifetime() method in BasicDatabase
that generates the right timestamp for each backend.
test/session_common.py:
added tests for more cases, get without default, getall non-existing
key etc. timestamp test changed to use new self.get_ts which is
overridden in other tests. Test that datatypes survive storage.
test/test_redis_session.py:
test redis session store with sqlite and anydbm primary databases
test/test_anydbm.py, test/test_sqlite.py
add test to make sure the databases are properly set up
sqlite - add test cases where anydbm is used as datastore
anydbm - remove updateTimestamp override add get_ts().
test/test_config.py
tests on redis_url and compatibility on choice of sessiondb backend
.travis.yml:
add redis db and redis-py
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Thu, 04 Aug 2022 14:41:58 -0400 |
| parents | 25d08e15e3b4 |
| children | feb175271f3e |
comparison
equal
deleted
inserted
replaced
| 6813:6b636fb29740 | 6814:3f60a71b0812 |
|---|---|
| 10 | 10 |
| 11 import os, marshal, shutil, time, logging | 11 import os, marshal, shutil, time, logging |
| 12 | 12 |
| 13 from roundup import hyperdb, date, password | 13 from roundup import hyperdb, date, password |
| 14 from roundup.backends import rdbms_common | 14 from roundup.backends import rdbms_common |
| 15 from roundup.backends.sessions_sqlite import Sessions, OneTimeKeys | 15 from roundup.backends import sessions_sqlite |
| 16 from roundup.backends import sessions_dbm | |
| 17 | |
| 18 try: | |
| 19 from roundup.backends import sessions_redis | |
| 20 except ModuleNotFoundError: | |
| 21 sessions_redis = None | |
| 22 | |
| 16 from roundup.anypy.strings import uany2s | 23 from roundup.anypy.strings import uany2s |
| 17 | 24 |
| 18 sqlite_version = None | 25 sqlite_version = None |
| 19 try: | 26 try: |
| 20 import sqlite3 as sqlite | 27 import sqlite3 as sqlite |
| 103 # We're using DBM for managing session info and one-time keys: | 110 # We're using DBM for managing session info and one-time keys: |
| 104 # For SQL database storage of this info we would need two concurrent | 111 # For SQL database storage of this info we would need two concurrent |
| 105 # connections to the same database which SQLite doesn't support | 112 # connections to the same database which SQLite doesn't support |
| 106 def getSessionManager(self): | 113 def getSessionManager(self): |
| 107 if not self.Session: | 114 if not self.Session: |
| 108 self.Session = Sessions(self) | 115 if self.config.SESSIONDB_BACKEND == "redis": |
| 116 if sessions_redis is None: | |
| 117 self.Session = sessions_sqlite.Sessions(self) | |
| 118 raise ValueError("[redis] session is set, but " | |
| 119 "redis module is not found") | |
| 120 self.Session = sessions_redis.Sessions(self) | |
| 121 elif self.config.SESSIONDB_BACKEND == "anydbm": | |
| 122 self.Session = sessions_dbm.Sessions(self) | |
| 123 else: | |
| 124 self.Session = sessions_sqlite.Sessions(self) | |
| 109 return self.Session | 125 return self.Session |
| 110 | 126 |
| 111 def getOTKManager(self): | 127 def getOTKManager(self): |
| 112 if not self.Otk: | 128 if not self.Otk: |
| 113 self.Otk = OneTimeKeys(self) | 129 if self.config.SESSIONDB_BACKEND == "redis": |
| 130 if sessions_redis is None: | |
| 131 self.Session = sessions_sqlite.OneTimeKeys(self) | |
| 132 raise ValueError("[redis] session is set, but " | |
| 133 "redis is not found") | |
| 134 self.Otk = sessions_redis.OneTimeKeys(self) | |
| 135 elif self.config.SESSIONDB_BACKEND == "anydbm": | |
| 136 self.Otk = sessions_dbm.OneTimeKeys(self) | |
| 137 else: | |
| 138 self.Otk = sessions_sqlite.OneTimeKeys(self) | |
| 114 return self.Otk | 139 return self.Otk |
| 115 | 140 |
| 116 def sqlite_busy_handler(self, data, table, count): | 141 def sqlite_busy_handler(self, data, table, count): |
| 117 """invoked whenever SQLite tries to access a database that is locked""" | 142 """invoked whenever SQLite tries to access a database that is locked""" |
| 118 now = time.time() | 143 now = time.time() |