diff doc/admin_guide.txt @ 6814:3f60a71b0812

Summary: Support selecion session/otk data store. Add redis as data store. Allow admin to select the backend data store. Compatibility matrix: main\/ session>| anydbm | sqlite | redis | mysql | postgresql | anydbm | D | | X | | | sqlite | X | D | X | | | mysql | | | | D | | postgresql | | | | | D | --------------------------------------------------------------+ D - default if unconfigured, X - compatible choice DETAILS roundup/configuration.py: add config.ini section sessiondb with settings: backend and redis_url. CHANGES.txt, doc/admin_guide.txt, doc/installation.txt, doc/upgrading.txt: doc on config of session db and redis. Plus some other fixes: admin - clarified why we do not drop __words and __testids table in native-fts conversion. TYpo fix. upgrading - doc how you can keep using anydbm for session data with sqlite. Fix dupe sentence in an upgrading config.ini section. roundup/backends/back_anydbm.py, roundup/backends/back_sqlite.py: code to support redis, redis/anydbm backends respectively. roundup/backends/sessions_redis.py new storage backend for redis. roundup/rest.py, roundup/cgi/actions.py, roundup/cgi/templating.py redis uses a different way of calculating lifetime/timestamp. Since expiration of an item occurred if its timestamp was more than 1 week old, code would calculate: now - 1 week + lifetime. But this results in faster expiration in redis if used for lifetime/timestamp. Convert code to use the lifetime() method in BasicDatabase that generates the right timestamp for each backend. test/session_common.py: added tests for more cases, get without default, getall non-existing key etc. timestamp test changed to use new self.get_ts which is overridden in other tests. Test that datatypes survive storage. test/test_redis_session.py: test redis session store with sqlite and anydbm primary databases test/test_anydbm.py, test/test_sqlite.py add test to make sure the databases are properly set up sqlite - add test cases where anydbm is used as datastore anydbm - remove updateTimestamp override add get_ts(). test/test_config.py tests on redis_url and compatibility on choice of sessiondb backend .travis.yml: add redis db and redis-py
author John Rouillard <rouilj@ieee.org>
date Thu, 04 Aug 2022 14:41:58 -0400
parents b3d4b25b4922
children 1319ab13f286
line wrap: on
line diff
--- a/doc/admin_guide.txt	Wed Aug 03 17:34:58 2022 -0400
+++ b/doc/admin_guide.txt	Thu Aug 04 14:41:58 2022 -0400
@@ -443,8 +443,94 @@
    delete from __words;
    delete from __textids;
 
-Note this deletes data from the tables and does *not* delete the
-table.
+Note this deletes data from the tables and does *not* delete
+the table. This allows you to revert to Roundup's native
+full text indexing on SQLite or Postgres. If you were to
+delete the tables, Roundup will not recreate the
+tables. Under PostgreSQL, you can use the ``truncate
+<tablename>`` command if you wish.
+
+Configuring Session Databases
+=============================
+
+The session and OTK (one time key) databases
+store information about the operation of Roundup.
+This ephemeral data:
+
+* web login session keys,
+* CSRF tokens,
+* email password recovery one time keys,
+* rate limiting data,
+* ...
+
+can be a performance bottleneck. It usually happens with
+anydbm or SQLite backends. PostgreSQL and MySQL are
+sufficiently powerful that they can handle the higher
+transaction rates.
+
+If you are using sqlite, you can choose to use the anydbm
+database for session data. By default it will use additional
+sqlite databases for storing the session and otk data.
+
+The following table shows which primary databases support
+different session database backends::
+
+
+  main\/ session>| anydbm | sqlite | redis | mysql | postgresql |
+  anydbm         |    D   |        |   X   |       |            |
+  sqlite         |    X   |    D   |   X   |       |            |
+  mysql          |        |        |       |   D   |            |
+  postgresql     |        |        |       |       |      D     |
+  --------------------------------------------------------------+
+        D - default if unconfigured,   X - compatible choice
+
+The ``backend`` setting is in the tracker's ``config.ini``
+file under the ``sessiondb`` section.
+
+Using Redis for Session Databases
+---------------------------------
+
+Redis is an in memory key/value data structure store.
+
+To use redis you must be using Python 3.6 or newer.  You
+need to install the redis-py_ module from pypi. Then install
+Redis using your package manager or by downloading it from
+the Redis_ website.
+
+You need to secure your redis instance. The data that
+Roundup stores includes session cookies and other
+authentication tokens. At minimum you should require a
+password to connect to your redis database. Set
+``requirepass`` in ``redis.conf``. Then change the
+``redis_url`` in ``config.ini`` to use the password.
+
+
+For example::
+
+   redis://:mypassword@localhost:7200/10
+
+will connect to the redis instance running on localhost at
+port 7200 using the password ``mypassword`` to open database
+10. The ``redis_url`` setting can load a file to better
+secure the url. If you are using redis 6.0 or newer, you can
+specify a username/password and access control lists to
+improv the security of your data. Another good alternative
+is to talk to redis using a Unix domain socket.
+
+If you are connecting to redis across the network rather
+than on localhost, you should configure ssl/tls and use the
+``rediss`` scheme in the url along with the query
+parameters::
+
+	ssl_cert_reqs=required&ssl_ca_certs=/path/to/custom/ca-cert
+
+where you specify the file that can be used to validate the
+SSL certificate. `Securing Redis`_ has more details.
+
+.. _Redis: https://redis.io
+.. _redis-py: https://pypi.org/project/redis/
+.. _Securing Redis: https://redis.io/docs/manual/security/
+
 
 Users and Security
 ==================
@@ -579,7 +665,7 @@
     python setup.py install
 
 6. Follow the steps in the `upgrading documentation`_ for all the
-   version between your original version and th new version.
+   versions between your original version and the new version.
 
    Usually you should run `roundup_admin -i <tracker_home> migrate`
    on your tracker(s) before you allow users to start accessing the tracker.

Roundup Issue Tracker: http://roundup-tracker.org/