diff doc/upgrading.txt @ 6604:0d99ae7c8de6

Allow Roundup to use PostgreSQL database native full text search back_postgreql.py - schema version changes for schema version 7. configuration.py - added indexer_language checks for postgresql. Hardcoded list for now. Docs admin_guide and upgrading Tests. This also restructures the version upgrade tests for the rdbms backends. They can run all of them now as the proper cascade is developed to roll back changes to version 6.
author John Rouillard <rouilj@ieee.org>
date Thu, 27 Jan 2022 19:48:48 -0500
parents 39189dd94f2c
children 2eec7a500333
line wrap: on
line diff
--- a/doc/upgrading.txt	Thu Jan 27 08:05:20 2022 +0100
+++ b/doc/upgrading.txt	Thu Jan 27 19:48:48 2022 -0500
@@ -94,29 +94,40 @@
 Enhanced full-text search (optional)
 ------------------------------------
 
-SQLite's `FTS5 full-text search engine`_ is available. It should work
-with any recent sqlite3 installation. Any full text search field will
-allow searching using the MATCH query format described at:
-https://www.sqlite.org/fts5.html#full_text_query_syntax. A list of
-words behaves almost the same as the default text search
+SQLite's `FTS5 full-text search engine`_ is available as is
+`PostgreSQL's full text search`_. Both require a schema upgrade so you
+should run::
+
+  roundup-admin -i tracker_home migrate
+
+to create FTS specific tables before restarting the roundup-web or
+email interfaces.
+
+SQLite 3.9.0+ or PostgreSQL 11.0+ are required to use this feature.
+When using SQLite, all full text search fields will allow searching
+using the MATCH query format described at:
+https://www.sqlite.org/fts5.html#full_text_query_syntax. When using
+PostgreSQL either the websearch_to_tsquery or to_tsquery formats
+described on
+https://www.postgresql.org/docs/14/textsearch-controls.html#TEXTSEARCH-PARSING-QUERIES
+can be used. The default is websearch. Prefixing the search with
+``ts:`` enables tsquery mode.
+
+A list of words behaves almost the same as the default text search
 (`native`). So the search string `fts search` will find all issues
 that have both of those words (an AND search) in a text-field (like
 title) or in a message (or file) attached to the issue.
 
-One thing to note is that the fts search does not ignore words longer
-than 25 characters or less than 2 characters. Also it does not filter
-out common works (i.e. there is no stoplist). So words like "and",
-"or", "then", "with" ...  are included in the FTS5 search. The native
-search applies both filters.
-
-Using SQLite FTS requires a schema change so you should run
-``roundup-admin -i tracker_home migrate`` as the FTS specific tables
-need to be created.
-
-Then you must explicitly enable it by changing the ``indexer`` setting
-in ``config.ini`` to ``native-fts``. Native-fts is never chosen by default
-like xapian or whoosh. This prevents the existing native indexing from
-being discarded if ``indexer`` is not set.
+One thing to note is that native-fts searches do not ignore words
+longer than 25 characters or less than 2 characters. Also SQLite does
+not filter out common works (i.e. there is no stopword list). So words
+like "and", "or", "then", "with" ...  are included in the FTS5 search.
+
+You must explicitly enable this search mechanism by changing the
+``indexer`` setting in ``config.ini`` to ``native-fts``. Native-fts is
+never chosen by default like xapian or whoosh. This prevents the
+existing native indexing from being discarded if ``indexer`` is not
+set.
 
 Next re-index your data with ``roundup-admin -i tracker_home
 reindex``. This can take a while depending on the size of the tracker.
@@ -124,14 +135,7 @@
 You may want to update your ``config.ini`` by following the directions
 above to get the latest documentation.
 
-If you are happy with the fts indexing, you can save some space by
-removing the data from the native text indexing tables. This requires
-using the ``sqlite3`` command to delete the rows in the ``__textids`` and
-``__words`` tables. You can do this with the following sqlite3
-commands::
-
-   delete from __words;
-   delete from __textids;
+See the `administration guide notes on native-fts`_ for further details.
 
 Adding error reporting templates (optional)
 -------------------------------------------
@@ -3362,4 +3366,6 @@
 .. _`administration guide`: admin_guide.html
 .. _`xmlrpc guide`: xmlrpc.html
 .. _FTS5 full-text search engine: https://www.sqlite.org/fts5.html
+.. _PostgreSQL's full text search: https://www.postgresql.org/docs/current/textsearch.html
+.. _`administration guide notes on native-fts`: admin_guide.html#configuring-native-fts-full-text-search
 .. _Configuring Compression: admin_guide.html#configuring-compression

Roundup Issue Tracker: http://roundup-tracker.org/