Mercurial > p > roundup > code
comparison doc/admin_guide.txt @ 6814:3f60a71b0812
Summary: Support selecion session/otk data store. Add redis as data store.
Allow admin to select the backend data store. Compatibility matrix:
main\/ session>| anydbm | sqlite | redis | mysql | postgresql |
anydbm | D | | X | | |
sqlite | X | D | X | | |
mysql | | | | D | |
postgresql | | | | | D |
--------------------------------------------------------------+
D - default if unconfigured, X - compatible choice
DETAILS
roundup/configuration.py:
add config.ini section sessiondb with settings: backend and redis_url.
CHANGES.txt, doc/admin_guide.txt, doc/installation.txt, doc/upgrading.txt:
doc on config of session db and redis. Plus some other fixes:
admin - clarified why we do not drop __words and __testids
table in native-fts conversion. TYpo fix.
upgrading - doc how you can keep using anydbm for session data with
sqlite. Fix dupe sentence in an upgrading config.ini
section.
roundup/backends/back_anydbm.py, roundup/backends/back_sqlite.py:
code to support redis, redis/anydbm backends respectively.
roundup/backends/sessions_redis.py
new storage backend for redis.
roundup/rest.py, roundup/cgi/actions.py, roundup/cgi/templating.py
redis uses a different way of calculating lifetime/timestamp.
Since expiration of an item occurred if its timestamp was more
than 1 week old, code would calculate:
now - 1 week + lifetime.
But this results in faster expiration in redis if used for
lifetime/timestamp.
Convert code to use the lifetime() method in BasicDatabase
that generates the right timestamp for each backend.
test/session_common.py:
added tests for more cases, get without default, getall non-existing
key etc. timestamp test changed to use new self.get_ts which is
overridden in other tests. Test that datatypes survive storage.
test/test_redis_session.py:
test redis session store with sqlite and anydbm primary databases
test/test_anydbm.py, test/test_sqlite.py
add test to make sure the databases are properly set up
sqlite - add test cases where anydbm is used as datastore
anydbm - remove updateTimestamp override add get_ts().
test/test_config.py
tests on redis_url and compatibility on choice of sessiondb backend
.travis.yml:
add redis db and redis-py
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Thu, 04 Aug 2022 14:41:58 -0400 |
| parents | b3d4b25b4922 |
| children | 1319ab13f286 |
comparison
equal
deleted
inserted
replaced
| 6813:6b636fb29740 | 6814:3f60a71b0812 |
|---|---|
| 441 the following SQL commands:: | 441 the following SQL commands:: |
| 442 | 442 |
| 443 delete from __words; | 443 delete from __words; |
| 444 delete from __textids; | 444 delete from __textids; |
| 445 | 445 |
| 446 Note this deletes data from the tables and does *not* delete the | 446 Note this deletes data from the tables and does *not* delete |
| 447 table. | 447 the table. This allows you to revert to Roundup's native |
| 448 full text indexing on SQLite or Postgres. If you were to | |
| 449 delete the tables, Roundup will not recreate the | |
| 450 tables. Under PostgreSQL, you can use the ``truncate | |
| 451 <tablename>`` command if you wish. | |
| 452 | |
| 453 Configuring Session Databases | |
| 454 ============================= | |
| 455 | |
| 456 The session and OTK (one time key) databases | |
| 457 store information about the operation of Roundup. | |
| 458 This ephemeral data: | |
| 459 | |
| 460 * web login session keys, | |
| 461 * CSRF tokens, | |
| 462 * email password recovery one time keys, | |
| 463 * rate limiting data, | |
| 464 * ... | |
| 465 | |
| 466 can be a performance bottleneck. It usually happens with | |
| 467 anydbm or SQLite backends. PostgreSQL and MySQL are | |
| 468 sufficiently powerful that they can handle the higher | |
| 469 transaction rates. | |
| 470 | |
| 471 If you are using sqlite, you can choose to use the anydbm | |
| 472 database for session data. By default it will use additional | |
| 473 sqlite databases for storing the session and otk data. | |
| 474 | |
| 475 The following table shows which primary databases support | |
| 476 different session database backends:: | |
| 477 | |
| 478 | |
| 479 main\/ session>| anydbm | sqlite | redis | mysql | postgresql | | |
| 480 anydbm | D | | X | | | | |
| 481 sqlite | X | D | X | | | | |
| 482 mysql | | | | D | | | |
| 483 postgresql | | | | | D | | |
| 484 --------------------------------------------------------------+ | |
| 485 D - default if unconfigured, X - compatible choice | |
| 486 | |
| 487 The ``backend`` setting is in the tracker's ``config.ini`` | |
| 488 file under the ``sessiondb`` section. | |
| 489 | |
| 490 Using Redis for Session Databases | |
| 491 --------------------------------- | |
| 492 | |
| 493 Redis is an in memory key/value data structure store. | |
| 494 | |
| 495 To use redis you must be using Python 3.6 or newer. You | |
| 496 need to install the redis-py_ module from pypi. Then install | |
| 497 Redis using your package manager or by downloading it from | |
| 498 the Redis_ website. | |
| 499 | |
| 500 You need to secure your redis instance. The data that | |
| 501 Roundup stores includes session cookies and other | |
| 502 authentication tokens. At minimum you should require a | |
| 503 password to connect to your redis database. Set | |
| 504 ``requirepass`` in ``redis.conf``. Then change the | |
| 505 ``redis_url`` in ``config.ini`` to use the password. | |
| 506 | |
| 507 | |
| 508 For example:: | |
| 509 | |
| 510 redis://:mypassword@localhost:7200/10 | |
| 511 | |
| 512 will connect to the redis instance running on localhost at | |
| 513 port 7200 using the password ``mypassword`` to open database | |
| 514 10. The ``redis_url`` setting can load a file to better | |
| 515 secure the url. If you are using redis 6.0 or newer, you can | |
| 516 specify a username/password and access control lists to | |
| 517 improv the security of your data. Another good alternative | |
| 518 is to talk to redis using a Unix domain socket. | |
| 519 | |
| 520 If you are connecting to redis across the network rather | |
| 521 than on localhost, you should configure ssl/tls and use the | |
| 522 ``rediss`` scheme in the url along with the query | |
| 523 parameters:: | |
| 524 | |
| 525 ssl_cert_reqs=required&ssl_ca_certs=/path/to/custom/ca-cert | |
| 526 | |
| 527 where you specify the file that can be used to validate the | |
| 528 SSL certificate. `Securing Redis`_ has more details. | |
| 529 | |
| 530 .. _Redis: https://redis.io | |
| 531 .. _redis-py: https://pypi.org/project/redis/ | |
| 532 .. _Securing Redis: https://redis.io/docs/manual/security/ | |
| 533 | |
| 448 | 534 |
| 449 Users and Security | 535 Users and Security |
| 450 ================== | 536 ================== |
| 451 | 537 |
| 452 Roundup holds its own user database which primarily contains a username, | 538 Roundup holds its own user database which primarily contains a username, |
| 577 5. Install the new version of the software:: | 663 5. Install the new version of the software:: |
| 578 | 664 |
| 579 python setup.py install | 665 python setup.py install |
| 580 | 666 |
| 581 6. Follow the steps in the `upgrading documentation`_ for all the | 667 6. Follow the steps in the `upgrading documentation`_ for all the |
| 582 version between your original version and th new version. | 668 versions between your original version and the new version. |
| 583 | 669 |
| 584 Usually you should run `roundup_admin -i <tracker_home> migrate` | 670 Usually you should run `roundup_admin -i <tracker_home> migrate` |
| 585 on your tracker(s) before you allow users to start accessing the tracker. | 671 on your tracker(s) before you allow users to start accessing the tracker. |
| 586 | 672 |
| 587 It's safe to run this even if it's not required, so just get into the | 673 It's safe to run this even if it's not required, so just get into the |