Mercurial > p > roundup > code
diff doc/admin_guide.txt @ 6814:3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
Allow admin to select the backend data store. Compatibility matrix:
main\/ session>| anydbm | sqlite | redis | mysql | postgresql |
anydbm | D | | X | | |
sqlite | X | D | X | | |
mysql | | | | D | |
postgresql | | | | | D |
--------------------------------------------------------------+
D - default if unconfigured, X - compatible choice
DETAILS
roundup/configuration.py:
add config.ini section sessiondb with settings: backend and redis_url.
CHANGES.txt, doc/admin_guide.txt, doc/installation.txt, doc/upgrading.txt:
doc on config of session db and redis. Plus some other fixes:
admin - clarified why we do not drop __words and __testids
table in native-fts conversion. TYpo fix.
upgrading - doc how you can keep using anydbm for session data with
sqlite. Fix dupe sentence in an upgrading config.ini
section.
roundup/backends/back_anydbm.py, roundup/backends/back_sqlite.py:
code to support redis, redis/anydbm backends respectively.
roundup/backends/sessions_redis.py
new storage backend for redis.
roundup/rest.py, roundup/cgi/actions.py, roundup/cgi/templating.py
redis uses a different way of calculating lifetime/timestamp.
Since expiration of an item occurred if its timestamp was more
than 1 week old, code would calculate:
now - 1 week + lifetime.
But this results in faster expiration in redis if used for
lifetime/timestamp.
Convert code to use the lifetime() method in BasicDatabase
that generates the right timestamp for each backend.
test/session_common.py:
added tests for more cases, get without default, getall non-existing
key etc. timestamp test changed to use new self.get_ts which is
overridden in other tests. Test that datatypes survive storage.
test/test_redis_session.py:
test redis session store with sqlite and anydbm primary databases
test/test_anydbm.py, test/test_sqlite.py
add test to make sure the databases are properly set up
sqlite - add test cases where anydbm is used as datastore
anydbm - remove updateTimestamp override add get_ts().
test/test_config.py
tests on redis_url and compatibility on choice of sessiondb backend
.travis.yml:
add redis db and redis-py
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Thu, 04 Aug 2022 14:41:58 -0400 |
| parents | b3d4b25b4922 |
| children | 1319ab13f286 |
line wrap: on
line diff
--- a/doc/admin_guide.txt Wed Aug 03 17:34:58 2022 -0400 +++ b/doc/admin_guide.txt Thu Aug 04 14:41:58 2022 -0400 @@ -443,8 +443,94 @@ delete from __words; delete from __textids; -Note this deletes data from the tables and does *not* delete the -table. +Note this deletes data from the tables and does *not* delete +the table. This allows you to revert to Roundup's native +full text indexing on SQLite or Postgres. If you were to +delete the tables, Roundup will not recreate the +tables. Under PostgreSQL, you can use the ``truncate +<tablename>`` command if you wish. + +Configuring Session Databases +============================= + +The session and OTK (one time key) databases +store information about the operation of Roundup. +This ephemeral data: + +* web login session keys, +* CSRF tokens, +* email password recovery one time keys, +* rate limiting data, +* ... + +can be a performance bottleneck. It usually happens with +anydbm or SQLite backends. PostgreSQL and MySQL are +sufficiently powerful that they can handle the higher +transaction rates. + +If you are using sqlite, you can choose to use the anydbm +database for session data. By default it will use additional +sqlite databases for storing the session and otk data. + +The following table shows which primary databases support +different session database backends:: + + + main\/ session>| anydbm | sqlite | redis | mysql | postgresql | + anydbm | D | | X | | | + sqlite | X | D | X | | | + mysql | | | | D | | + postgresql | | | | | D | + --------------------------------------------------------------+ + D - default if unconfigured, X - compatible choice + +The ``backend`` setting is in the tracker's ``config.ini`` +file under the ``sessiondb`` section. + +Using Redis for Session Databases +--------------------------------- + +Redis is an in memory key/value data structure store. + +To use redis you must be using Python 3.6 or newer. You +need to install the redis-py_ module from pypi. Then install +Redis using your package manager or by downloading it from +the Redis_ website. + +You need to secure your redis instance. The data that +Roundup stores includes session cookies and other +authentication tokens. At minimum you should require a +password to connect to your redis database. Set +``requirepass`` in ``redis.conf``. Then change the +``redis_url`` in ``config.ini`` to use the password. + + +For example:: + + redis://:mypassword@localhost:7200/10 + +will connect to the redis instance running on localhost at +port 7200 using the password ``mypassword`` to open database +10. The ``redis_url`` setting can load a file to better +secure the url. If you are using redis 6.0 or newer, you can +specify a username/password and access control lists to +improv the security of your data. Another good alternative +is to talk to redis using a Unix domain socket. + +If you are connecting to redis across the network rather +than on localhost, you should configure ssl/tls and use the +``rediss`` scheme in the url along with the query +parameters:: + + ssl_cert_reqs=required&ssl_ca_certs=/path/to/custom/ca-cert + +where you specify the file that can be used to validate the +SSL certificate. `Securing Redis`_ has more details. + +.. _Redis: https://redis.io +.. _redis-py: https://pypi.org/project/redis/ +.. _Securing Redis: https://redis.io/docs/manual/security/ + Users and Security ================== @@ -579,7 +665,7 @@ python setup.py install 6. Follow the steps in the `upgrading documentation`_ for all the - version between your original version and th new version. + versions between your original version and the new version. Usually you should run `roundup_admin -i <tracker_home> migrate` on your tracker(s) before you allow users to start accessing the tracker.
