view doc/mysql.txt @ 5710:0b79bfcb3312

Add support for making an idempotent POST. This allows retrying a POST that was interrupted. It involves creating a post once only (poe) url /rest/data/<class>/@poe/<random_token>. This url acts the same as a post to /rest/data/<class>. However once the @poe url is used, it can't be used for a second POST. To make these changes: 1) Take the body of post_collection into a new post_collection_inner function. Have post_collection call post_collection_inner. 2) Add a handler for POST to rest/data/class/@poe. This will return a unique POE url. By default the url expires after 30 minutes. The POE random token is only good for a specific user and is stored in the session db. 3) Add a handler for POST to rest/data/<class>/@poe/<random token>. The random token generated in 2 is validated for proper class (if token is not generic) and proper user and must not have expired. If everything is valid, call post_collection_inner to process the input and generate the new entry. To make recognition of 2 stable (so it's not confused with rest/data/<:class_name>/<:item_id>), removed @ from Routing::url_to_regex. The current Routing.execute method stops on the first regular expression to match the URL. Since item_id doesn't accept a POST, I was getting 405 bad method sometimes. My guess is the order of the regular expressions is not stable, so sometime I would get the right regexp for /data/<class>/@poe and sometime I would get the one for /data/<class>/<item_id>. By removing the @ from the url_to_regexp, there was no way for the item_id case to match @poe. There are alternate fixes we may need to look at. If a regexp matches but the method does not, return to the regexp matching loop in execute() looking for another match. Only once every possible match has failed should the code return a 405 method failure. Another fix is to implement a more sophisticated mechanism so that @Routing.route("/data/<:class_name>/<:item_id>/<:attr_name>", 'PATCH') has different regexps for matching <:class_name> <:item_id> and <:attr_name>. Currently the regexp specified by url_to_regex is used for every component. Other fixes: Made failure to find any props in props_from_args return an empty dict rather than throwing an unhandled error. Make __init__ for SimulateFieldStorageFromJson handle an empty json doc. Useful for POSTing to rest/data/class/@poe with an empty document. Testing: added testPostPOE to test/rest_common.py that I think covers all the code that was added. Documentation: Add doc to rest.txt in the "Client API" section titled: Safely Re-sending POST". Move existing section "Adding new rest endpoints" in "Client API" to a new second level section called "Programming the REST API". Also a minor change to the simple rest client moving the header setting to continuation lines rather than showing one long line.
author John Rouillard <rouilj@ieee.org>
date Sun, 14 Apr 2019 21:07:11 -0400
parents 0df5f9eeefd4
children e48b039b0ec0
line wrap: on
line source

=============
MySQL Backend
=============

This notes detail the MySQL backend for the Roundup issue tracker.


Prerequisites
=============

To use MySQL as the backend for storing roundup data, you also need 
to install:

1. MySQL RDBMS 4.0.18 or higher - http://www.mysql.com. Your MySQL
   installation MUST support InnoDB tables (or Berkeley DB (BDB) tables
   if you have no other choice). If you're running < 4.0.18 (but not <4.0)
   then you'll need to use BDB to pass all unit tests. Edit the
   ``roundup/backends/back_mysql.py`` file to enable DBD instead of InnoDB.
2. Python MySQL interface - https://pypi.org/project/mysqlclient/


Other Configuration
===================

If you are indexing large documents (e.g attached file contents)
using MySQL, you may need to increase the max_allowed_packet size.
If you don't you can see the error::

   'MySql Server has gone away (2006)'

To do this edit /etc/my.conf and change::

    [mysqld]
    max_allowed_packet = 1M

the 'max_allowed_packet' value from '1M' to '64M' or
larger.

Alternatively you can install an alternate indexer (whoosh, xapian
etc.) and force the tracker to use it by setting the ``indexer``
setting in the tracker's ``config.ini``.

This fix was supplied by telsch. See issue
https://issues.roundup-tracker.org/issue2550743 for further info or if
you are interested in developing a patch to roundup to help work
around this issue.

Running the MySQL tests
=======================

Roundup tests expect an empty MySQL database. Two alternate ways to provide 
this:

1. If you have root permissions on the MySQL server, you can create 
   the necessary database entries using the follwing SQL sequence. Use
   ``mysql`` on the command line to enter::

       CREATE DATABASE rounduptest;
       USE rounduptest;
       GRANT ALL PRIVILEGES ON rounduptest.* TO rounduptest@localhost
            IDENTIFIED BY 'rounduptest';
       FLUSH PRIVILEGES;

2. If your administrator has provided you with database connection info, 
   see the config values in 'test/db_test_base.py' 
   about which database connection, name and user will be used.

The MySQL database should not contain any tables. Tests will not 
drop the database with existing data.


Showing MySQL who's boss
========================

If things ever get to the point where that test database is totally hosed,
just::

  $ su -
  # /etc/init.d/mysql stop
  # rm -rf /var/lib/mysql/rounduptest
  # /etc/init.d/mysql start

and all will be better (note that on some systems, ``mysql`` is spelt
``mysqld``).


Roundup Issue Tracker: http://roundup-tracker.org/