Mercurial > p > roundup > code

comparison doc/upgrading.txt @ 6814:3f60a71b0812

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Summary: Support selecion session/otk data store. Add redis as data store. Allow admin to select the backend data store. Compatibility matrix: main\/ session>| anydbm | sqlite | redis | mysql | postgresql | anydbm | D | | X | | | sqlite | X | D | X | | | mysql | | | | D | | postgresql | | | | | D | --------------------------------------------------------------+ D - default if unconfigured, X - compatible choice DETAILS roundup/configuration.py: add config.ini section sessiondb with settings: backend and redis_url. CHANGES.txt, doc/admin_guide.txt, doc/installation.txt, doc/upgrading.txt: doc on config of session db and redis. Plus some other fixes: admin - clarified why we do not drop __words and __testids table in native-fts conversion. TYpo fix. upgrading - doc how you can keep using anydbm for session data with sqlite. Fix dupe sentence in an upgrading config.ini section. roundup/backends/back_anydbm.py, roundup/backends/back_sqlite.py: code to support redis, redis/anydbm backends respectively. roundup/backends/sessions_redis.py new storage backend for redis. roundup/rest.py, roundup/cgi/actions.py, roundup/cgi/templating.py redis uses a different way of calculating lifetime/timestamp. Since expiration of an item occurred if its timestamp was more than 1 week old, code would calculate: now - 1 week + lifetime. But this results in faster expiration in redis if used for lifetime/timestamp. Convert code to use the lifetime() method in BasicDatabase that generates the right timestamp for each backend. test/session_common.py: added tests for more cases, get without default, getall non-existing key etc. timestamp test changed to use new self.get_ts which is overridden in other tests. Test that datatypes survive storage. test/test_redis_session.py: test redis session store with sqlite and anydbm primary databases test/test_anydbm.py, test/test_sqlite.py add test to make sure the databases are properly set up sqlite - add test cases where anydbm is used as datastore anydbm - remove updateTimestamp override add get_ts(). test/test_config.py tests on redis_url and compatibility on choice of sessiondb backend .travis.yml: add redis db and redis-py
author John Rouillard <rouilj@ieee.org>
date Thu, 04 Aug 2022 14:41:58 -0400
parents bdd28b244839
children 1319ab13f286
comparison
equal deleted inserted replaced
6813:6b636fb29740 6814:3f60a71b0812
32 32
33 .. index:: Upgrading; 2.2.0 to 2.3.0 33 .. index:: Upgrading; 2.2.0 to 2.3.0
34 34
35 Migrating from 2.2.0 to 2.3.0 35 Migrating from 2.2.0 to 2.3.0
36 ============================= 36 =============================
37
38 Update your ``config.ini`` (required)
39 -------------------------------------
40
41 Upgrade tracker's config.ini file. Use::
42
43 roundup-admin -i /path/to/tracker updateconfig newconfig.ini
44
45 to generate a new ini file preserving all your settings.
46 You can then merge any local comments from the tracker's
47 ``config.ini`` to ``newconfig.ini`` and replace
48 ``config.ini`` with ``newconfig.ini``.
37 49
38 Rdbms version change from 7 to 8 (required) 50 Rdbms version change from 7 to 8 (required)
39 ------------------------------------------- 51 -------------------------------------------
40 52
41 This release includes a change that requires updates to the 53 This release includes a change that requires updates to the
65 "Tracker updated" (if you've not previously run it on an 77 "Tracker updated" (if you've not previously run it on an
66 RDBMS backend) or "No migration action required" (if you 78 RDBMS backend) or "No migration action required" (if you
67 have run it, or have used another interface to the tracker, 79 have run it, or have used another interface to the tracker,
68 or are using anydbm). 80 or are using anydbm).
69 81
70 Session/OTK data storage for SQLite backend changed 82 Session/OTK data storage for SQLite backend changed (required)
71 --------------------------------------------------- 83 --------------------------------------------------------------
72 84
73 Roundup stores a lot of ephemeral data: 85 Roundup stores a lot of ephemeral data:
74 86
75 * login session tokens, 87 * login session tokens,
76 * rate limits 88 * rate limits
81 These were stored using dbm style files while the main data 93 These were stored using dbm style files while the main data
82 is stored in a SQLite db. Using both dbm and sqlite style 94 is stored in a SQLite db. Using both dbm and sqlite style
83 files is surprising and due to how we lock dbm files can be 95 files is surprising and due to how we lock dbm files can be
84 a performance issue. 96 a performance issue.
85 97
86 In this release two sqlite databases called ``db-otk`` and 98 However you can continue to use the dbm files by setting the
87 ``db-session`` replace the dbm databases. Once you make the 99 ``backend`` option in the ``[sessiondb]`` section of
88 change the old ``otks`` and ``sessions`` dbm databases can 100 ``config.ini`` to ``anydbm``.
89 be removed. 101
102 If you do not change the setting, two sqlite databases
103 called ``db-otk`` and ``db-session`` replace the dbm
104 databases. Once you make the change the old ``otks`` and
105 ``sessions`` dbm databases can be removed.
90 106
91 Note this replacement will require users to log in again and 107 Note this replacement will require users to log in again and
92 refresh web pages to save data. It is best is people save 108 refresh web pages to save data. It is best if people save
93 all their changes and log out of Roundup before the upgrade 109 all their changes and log out of Roundup before the upgrade
94 is done to minimize confusion. Because the data is 110 is done to minimize confusion. Because the data is
95 ephemeral, there is no plan to migrate this data to the new 111 ephemeral, there is no plan to migrate this data to the new
96 SQLite databases. 112 SQLite databases. If you want to keep using the data set the
113 ``sessiondb`` ``backend`` option as described above.
114
115 Session/OTK data storage using Redis (optional)
116 -----------------------------------------------
117
118 If you are running Roundup with Python 3.6 or newer, you can
119 store your ephemeral data in a Redis database. This provides
120 significantly better performance for ephemeral data than
121 SQLite or dbm files. See the section `Using Redis for Session
122 Databases`_ in the `administration guide`_
123
124
125 .. _Using Redis for Session Databases:
126 admin_guide.html#using-redis-for-session-databases
97 127
98 .. index:: Upgrading; 2.1.0 to 2.2.0 128 .. index:: Upgrading; 2.1.0 to 2.2.0
99 129
100 Migrating from 2.1.0 to 2.2.0 130 Migrating from 2.1.0 to 2.2.0
101 ============================= 131 =============================
105 135
106 Upgrade tracker's config.ini file. Use:: 136 Upgrade tracker's config.ini file. Use::
107 137
108 roundup-admin -i /path/to/tracker updateconfig newconfig.ini 138 roundup-admin -i /path/to/tracker updateconfig newconfig.ini
109 139
110 to generate a new ini file preserving all your settings. You 140 to generate a new ini file preserving all your settings.
111 can then merge any local comments from the tracker's 141 You can then merge any local comments from the tracker's
112 ``config.ini`` into ``newconfig.ini``. You can then merge 142 ``config.ini`` to ``newconfig.ini`` and replace
113 comments from ``config.ini`` to ``newconfig.ini`` and 143 ``config.ini`` with ``newconfig.ini``.
114 replace ``config.ini`` with ``newconfig.ini``.
115 144
116 Rdbms version change from 6 to 7 (required) 145 Rdbms version change from 6 to 7 (required)
117 ------------------------------------------- 146 -------------------------------------------
118 147
119 This release includes two changes that require updates to the database 148 This release includes two changes that require updates to the database
Roundup Issue Tracker: http://roundup-tracker.org/