Mercurial > p > roundup > code

comparison doc/upgrading.txt @ 6588:91ab3e0ffcd0

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Summary: Add test cases for sqlite fts Add support for using the FTS5 full text query engine for sqlite. Also stubbed out some sections for adding postgresql FTS support as well. Added nee indexer type native-fts. It is not selected by default. The indexer=native is used if no indexer is set. This prevents an upgrade from seeming to wipe out the native index if upgraded and indexer=native is not explicitly set. Docs updated. Also changed section headers to sentence case for the current release notes. Indexing backend can control if the full text search phrase is broken into a list of words or passed intact. For backends with query languages (sqlite and can be enabled for whoosh and xapian) we do not want the phrase "tokenized" on whitespace. This also updates the rdbms database version to version 7 to add FTS table. I will be using the same version when I add postgresql. If somebody runs this version on postgresql, they will have to manually add the fts tables for postgresql if they want to use it. Added a new renderError method to client. This allows errors to be reported still using page.html rather than raw html. It also supports templates for any error code. If no template for the error code (e.g. 400) is found, the error in raw html with no page frame is shown. New IndexerQueryError exception to pass back message about query syntax errors.
author John Rouillard <rouilj@ieee.org>
date Sun, 23 Jan 2022 18:57:45 -0500
parents 24e2eeb2ed9a
children 5ce396880899
comparison
equal deleted inserted replaced
6587:4f8fc55730e1 6588:91ab3e0ffcd0
33 .. index:: Upgrading; 2.0.0 to 2.1.0 33 .. index:: Upgrading; 2.0.0 to 2.1.0
34 34
35 Migrating from 2.1.0 to 2.x.y 35 Migrating from 2.1.0 to 2.x.y
36 ============================= 36 =============================
37 37
38 Check Compression Settings 38 Check compression settings
39 -------------------------- 39 --------------------------
40 40
41 Read the `administration guide`_ section on 'Configuring Compression'. 41 Read the `administration guide`_ section on 'Configuring Compression'.
42 42
43 Upgrade tracker's config.ini file. Use:: 43 Upgrade tracker's config.ini file. Use::
48 merge any local comments from the tracker's ``config.ini`` into 48 merge any local comments from the tracker's ``config.ini`` into
49 ``newconfig.ini``. Compare the old and new files and configure new 49 ``newconfig.ini``. Compare the old and new files and configure new
50 compression settings as you want. Then replace ``config.ini`` with the 50 compression settings as you want. Then replace ``config.ini`` with the
51 ``newconfig.ini`` file. 51 ``newconfig.ini`` file.
52 52
53 Search Added to User Index Page 53 Search added to user index page
54 ------------------------------- 54 -------------------------------
55 55
56 A search form and count of number of hits has been added to the 56 A search form and count of number of hits has been added to the
57 ``user.index.html`` template page in the classic template. You may 57 ``user.index.html`` template page in the classic template. You may
58 want to merge the search form and footer into your template. 58 want to merge the search form and footer into your template.
59
60 Enhanced full-text search
61 -------------------------
62
63 SQLite's `FTS5 full-text search engine`_ is available. It should work
64 with any recent sqlite3 installation. Any full text search field will
65 allow searching using the MATCH query format described at:
66 https://www.sqlite.org/fts5.html#full_text_query_syntax. A list of
67 words behaves almost the same as the default text search
68 (`native`). So the search string `fts search` will find all issues
69 that have both of those words (an AND search) in a text-field (like
70 title) or in a message (or file) attached to the issue.
71
72 One thing to note is that the fts search does not ignore words longer
73 than 25 characters or less than 2 characters. Also it does not filter
74 out common works (i.e. there is no stoplist). So words like "and",
75 "or", "then", "with" ... are included in the FTS5 search. The native
76 search applies both filters.
77
78 Using SQLite FTS requires a schema change so you should run
79 `roundup-admin -i tracker_home migrate` as the FTS specific tables
80 need to be created.
81
82 Then you must explicitly enable it by changing the `indexer` setting
83 in `config.ini` to `native-fts`. Native-fts is never chosen by default
84 like xapian or whoosh. This prevents the existing native indexing from
85 being discarded if `indexer` is not set.
86
87 Next re-index your data with `roundup-admin -i tracker_home
88 reindex`. This can take a while depending on the size of the tracker.
89
90 You may want to update your `config.ini` by following the directions
91 above to get the latest documentation.
92
93 If you are happy with the fts indexing, you can save some space by
94 removing the data from the native text indexing tables. This requires
95 using the `sqlite3` command to delete the rows in the `__textids` and
96 `__words` tables. You can do this with the following sqlite3
97 commands::
98
99 delete from __words;
100 delete from __textids;
59 101
60 102
61 Migrating from 2.0.0 to 2.1.0 103 Migrating from 2.0.0 to 2.1.0
62 ============================= 104 =============================
63 105
3268 3310
3269 .. _`customisation documentation`: customizing.html 3311 .. _`customisation documentation`: customizing.html
3270 .. _`security documentation`: security.html 3312 .. _`security documentation`: security.html
3271 .. _`administration guide`: admin_guide.html 3313 .. _`administration guide`: admin_guide.html
3272 .. _`xmlrpc guide`: xmlrpc.html 3314 .. _`xmlrpc guide`: xmlrpc.html
3315 .. _FTS5 full-text search engine: https://www.sqlite.org/fts5.html
Roundup Issue Tracker: http://roundup-tracker.org/