view doc/features.txt @ 6915:9ff091537f43

postgresql native-fts; more indexer tests 1) Make postgresql native-fts actually work. 2) Add simple stopword filtering to sqlite native-fts indexer. 3) Add more tests for indexer_common get_indexer Details: 1) roundup/backends/indexer_postgresql_fts.py: ignore ValueError raised if we try to index a string with a null character in it. This could happen due to an incorrect text/ mime type on a file that has nulls in it. Replace ValueError raised by postgresql with customized IndexerQueryError if a search string has a null in it. roundup/backends/rdbms_common.py: Make postgresql native-fts work. When specified it was using using whatever was returned from get_indexer(). However loading the native-fts indexer backend failed because there was no connection to the postgresql database when this call was made. Simple solution, move the call after the open_connection call in Database::__init__(). However the open_connection call creates the schema for the database if it is not there. The schema builds tables for indexer=native type indexing. As part of the build it looks at the indexer to see the min/max size of the indexed tokens. No indexer define, we get a crash. So it's a a chicken/egg issue. I solved it by setting the indexer to the Indexer from indexer_common which has the min/max token size info. I also added a no-op save_indexer to this Indexer class. I claim save_indexer() isn't needed as a commit() on the db does all the saving required. Then after open_connection is called, I call get_indexer to retrieve the correct indexer and indexer_postgresql_fts woks since the conn connection property is defined. roundup/backends/indexer_common.py: add save_index() method for indexer. It does nothing but is needed in rdbms backends during schema initialization. 2) roundup/backends/indexer_sqlite_fts.py: when this indexer is used, the indexer test in DBTest on the word "the" fail. This is due to missing stopword filtering. Implement basic stopword filtering for bare stopwords (like 'the') to make the test pass. Note: this indexer is not currently automatically run by the CI suite, it was found during manual testing. However there is a FIXME to extract the indexer tests from DBTest and run it using this backend. roundup/configuration.py, roundup/doc/admin_guide.txt: update doc on stopword use for sqlite native-fts. test/db_test_base.py: DBTest::testStringBinary creates a file with nulls in it. It was breaking postgresql with native-fts indexer. Changed test to assign mime type application/octet-stream that prevents it from being processed by any text search indexer. add test to exclude indexer searching in specific props. This code path was untested before. test/test_indexer.py: add test to call find with no words. Untested code path. add test to index and find a string with a null \x00 byte. it was tested inadvertently by testStringBinary but this makes it explicit and moves it to indexer testing. (one version each for: generic, postgresql and mysql) Renamed Get_IndexerAutoSelectTest to Get_IndexerTest and renamed autoselect tests to include autoselect. Added tests for an invalid indexer and using native-fts with anydbm (unsupported combo) to make sure the code does something useful if the validation in configuration.py is broken. test/test_liveserver.py: add test to load an issue add test using text search (fts) to find the issue add tests to find issue using postgresql native-fts test/test_postgresql.py, test/test_sqlite.py: added explanation on how to setup integration test using native-fts. added code to clean up test environment if native-fts test is run.
author John Rouillard <rouilj@ieee.org>
date Mon, 05 Sep 2022 16:25:20 -0400
parents e7b4ad2c57ac
children 37e2e3897ddf
line wrap: on
line source

.. meta::
    :description:
        Features of using the Roundup Issue Tracker. Describes
        all access methods, configuration and workflow capabilities.
	Links to detailed documentation.

================
Roundup Features
================

Roundup is a simple-to-use and -install issue-tracking system with
web, e-mail and command-line interfaces. It is based on the winning design
from Ka-Ping Yee in the :index:`Software Carpentry` "Track" design competition.

*simple to install*
 - installation (including web interface) takes about 30 minutes
 - instant-gratification ``python demo.py`` :)
 - two templates included in the distribution for you to base your tracker on
 - play with the demo, customise it and then use *it* as the template for
   your production tracker
 - requires *no* additional support software - python (2.7 or 3.4+) is
   enough to get you going
 - easy to set up higher-performance storage backends like sqlite_,
   mysql_ and postgresql_

*simple to use*
 - accessible through the web (html, REST or XMLRPC), email,
   command-line or Python programs
 - may be used to track bugs, features, user feedback, sales opportunities,
   milestones, ...
 - automatically keeps a full history of changes to issues with
   configurable verbosity and easy access to information about who created
   or last modified *any* item in the database
 - issues have their own mini mailing list (nosy list)
 - users may sign themselves up, there may be automatic signup for
   incoming email and users may handle their own password reset requests

*highly configurable*
 - web interface HTML is fully editable
 - database schema is also fully editable (only the "user" class is required)
   with a full set of data types (including dates and many-to-many relations)
   across all storages available
 - customised automatic auditors and reactors may be written that perform
   actions before and after changes are made to entries in the database,
   or may veto the creation or modification of items in the database
 - samples are provided for all manner of configuration changes and
   customisations

*fast, scalable*
 - with the sqlite, mysql and postgresql backends, roundup is
   also fast and scalable, easily handling thousands of issues and users
   with decent response times
 - database indexes are automatically added for those backends that
   support them (sqlite, mysql and postgresql)
 - indexed text searching giving fast responses to searches across all
   messages and indexed string properties
 - support for the Xapian, Whoosh. sqlite or postgres full-text
   indexing engines for large trackers

*documented*
 - documentation exists for installation, upgrading, maintenance, users and
   customisation

*web interface*
 - fully editable interfaces for listing and display of items
 - extendable to include wizards, parent/meta bug displays, ...
 - differentiates between anonymous, known and admin users
 - may be set up to require login, and may also only allow admin users
   to register new users
 - authentication of user registration and user-driven password resetting
   using email and one time keys
 - may be run using WSGI or through CGI as a normal cgi script, as a
   stand-alone web server, under mod_wsgi (mod_python is deprecated) or through Zope
 - searching may be performed using many constraints, including a full-text
   search of messages attached to issues
 - file attachments (added through the web or email) are served up with the
   correct content-type and filename
 - email change messages generated by roundup appear to be sent by the
   person who made the change, but responses will go back through the nosy
   list by default
 - flexible access control built around Permissions and Roles with assigned
   Permissions
 - generates valid HTML4 or XHTML
 - detects concurrent user changes
 - saving and editing of user-defined queries which may optionally be
   shared with other users

*e-mail interface*
 - may be set up using sendmail-like delivery alias, POP polling or mailbox
   polling
 - may auto-register users who send in mail and aren't known to roundup
 - nosy list configuration controls how people are added and when messages
   are sent to the list
 - auto-filing of "unformatted" messages into a particular class
 - e-mail attachments are handled sanely, being attached to the issue they're
   intended for, and forwarded on to the nosy list
 - sane handling of content-type and content-encoding of messages (text/plain
   preferred in all situations)
 - email packages that display threading will show issue messages correctly
   threaded
 - users may send in messages from multiple addresses and be associated
   with the same roundup username
 - built-in security features like TLS and APOP

*command-line*
 - may be used to interactively manage roundup databases
 - may be scripted using standard shell scripting
 - roundup's API may also be used by other Python programs - a sample is
   provided that runs once a day and emails people their assigned issues
 - a variety of sample shell scripts are provided (weekly reports, issue
   generation, ...)

*xmlrpc interface*
 - simple remote tracker interface with basic HTTP authentication
   available at the /xmlrpc endpoint.
 - provides same access to tracker as roundup-admin, but based on
   XMLRPC calls
 - see the `xmlrpc guide`_ for more details simple clients etc.

*RESTful interface*
 - accessible using basic HTTP authentication at /rest starting point
 - able to be extended with new endpoints
 - authentication can be done using JSON Web Tokens (JWT)
 - CORS support for use by third party web pages.
 - see the `rest guide`_ for details.

.. _sqlite: http://www.hwaci.com/sw/sqlite/
.. _mysql: https://pypi.org/project/MySQL-python/
.. _postgresql: https://www.psycopg.org/
.. _`xmlrpc guide`: xmlrpc.html
.. _`rest guide`: rest.html

Roundup Issue Tracker: http://roundup-tracker.org/