Mercurial Repository: p/roundup/code: doc/customizing.txt comparison

comparison doc/customizing.txt @ 7464:82bbb95e5690 issue2550923_computed_property

merge from tip into issue2550923_computed_property

author	John Rouillard <rouilj@ieee.org>
date	Thu, 08 Jun 2023 00:10:32 -0400
parents	ca90f7270cd4 a1d0b505766e
children	14a8e11f3a87

comparison

equal deleted inserted replaced

-:ca90f7270cd4
+:82bbb95e5690
 .. meta::
 :description:
 How to customize and extend the Roundup Issue
 Tracker. Includes many cookbook and how-to
-examples. Reference for the design and internals
+examples.
-needed to understand and extend the examples to meet
-new needs.
 :tocdepth: 2
 ===================
 Customising Roundup
 ===================
 .. This document borrows from the ZopeBook section on ZPT. The original is at:
 http://www.zope.org/Documentation/Books/ZopeBook/current/ZPT.stx
+.. admonition:: Welcome
+This document used to be much larger and include a lot of
+reference material. That has been moved to the `reference document`_.
+The documentation is slowly being reorganized using the `Diataxis
+framework`_. Help with the reorganization is welcome.
+.. _reference document: reference.html
+.. _diataxis framework: https://diataxis.fr/
 .. contents::
-:depth: 2
+:depth: 3
 :local:
 What You Can Do
 ===============
 Before you get too far, it's probably worth having a quick read of the Roundup
 `design documentation`_.
 Customisation of Roundup can take one of six forms:
-1. `tracker configuration`_ changes
+1. `tracker configuration <reference.html#tracker-configuration>`_ changes
-2. database, or `tracker schema`_ changes
+2. database, or `tracker schema <reference.html#tracker-schema>`_ changes
-3. "definition" class `database content`_ changes
+3. "definition" class `database content <reference.html#database-content>`_ changes
-4. behavioural changes through detectors_, extensions_ and interfaces.py_
+4. behavioural changes through `detectors <reference.html#detectors>`_,
-5. `security / access controls`_
+`extensions <reference.html#extensions>`_ and
-6. change the `web interface`_
+`interfaces.py <reference.html#interfaces-py>`_
+5. `security / access controls <reference.html#security-access-controls>`_
+6. change the `web interface <reference.html#web-interface>`_
 The third case is special because it takes two distinctly different forms
 depending upon whether the tracker has been initialised or not. The other two
 may be done at any time, before or after tracker initialisation. Yes, this
 includes adding or removing properties from classes.
+.. _CustomExamples:
-Trackers in a Nutshell
-======================
+Examples
+========
-Trackers have the following structure:
+Changing what's stored in the database
-.. index::
+--------------------------------------
-single: tracker; structure db directory
-single: tracker; structure detectors directory
+The following examples illustrate ways to change the information stored in
-single: tracker; structure extensions directory
+the database.
-single: tracker; structure html directory
-single: tracker; structure html directory
-single: tracker; structure lib directory
+Adding a new field to the classic schema
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-=================== ========================================================
-Tracker File        Description
+This example shows how to add a simple field (a due date) to the default
-=================== ========================================================
+classic schema. It does not add any additional behaviour, such as enforcing
-config.ini          Holds the basic `tracker configuration`_
+the due date, or causing automatic actions to fire if the due date passes.
-schema.py           Holds the `tracker schema`_
-initial_data.py     Holds any data to be entered into the database when the
+You add new fields by editing the ``schema.py`` file in you tracker's home.
-tracker is initialised.
+Schema changes are automatically applied to the database on the next
-db/                 Holds the tracker's database
+tracker access (note that roundup-server would need to be restarted as it
-db/files/           Holds the tracker's upload files and messages
+caches the schema).
-db/backend_name     Names the database back-end for the tracker (obsolete).
-Current way uses the ``backend`` setting in the rdbms
+.. index:: schema; example changes
-		    section of config.ini.
-detectors/          Auditors and reactors for this tracker
+1. Modify the ``schema.py``::
-extensions/         Additional actions and `templating utilities`_
-html/               Web interface templates, images and style sheets
+issue = IssueClass(db, "issue",
-lib/                optional common imports for detectors and extensions
+assignedto=Link("user"), keyword=Multilink("keyword"),
-=================== ========================================================
+priority=Link("priority"), status=Link("status"),
+due_date=Date())
-.. index:: config.ini
+2. Add an edit field to the ``issue.item.html`` template::
-.. index:: configuration; see config.ini
+<tr>
-Tracker Configuration
+<th>Due Date</th>
-=====================
+<td tal:content="structure context/due_date/field" />
+</tr>
-The ``config.ini`` located in your tracker home contains the basic
-configuration for the web and e-mail components of roundup's interfaces.
+If you want to show only the date part of due_date then do this instead::
-Changes to the data captured by your tracker is controlled by the `tracker
-schema`_.  Some configuration is also performed using permissions - see the
-`security / access controls`_ section. For example, to allow users to
-automatically register through the email interface, you must grant the
-"Anonymous" Role the "Email Access" Permission.
-.. index::
-single: config.ini; sections
-see: configuration; config.ini
-The following is taken from the `Python Library Reference`__ (July 18, 2018)
-section "ConfigParser -- Configuration file parser":
-The configuration file consists of sections, led by a [section] header
-and followed by name: value entries, with continuations in the style
-of RFC 822 (see section 3.1.1, “LONG HEADER FIELDS”); name=value is
-also accepted. Note that leading whitespace is removed from
-values. The optional values can contain format strings which refer to
-other values in the same section, or values in a special DEFAULT
-section. Additional defaults can be provided on initialization and
-retrieval. Lines beginning with '#' or ';' are ignored and may be
-used to provide comments.
-For example::
-[My Section]
-foodir = %(dir)s/whatever
-dir = frob
-would resolve the "%(dir)s" to the value of "dir" ("frob" in this case)
-resulting in "foodir" being "frob/whatever".
-__ https://docs.python.org/2/library/configparser.html
-Example configuration settings are below. This is a partial
-list. Documentation on all the settings is included in the
-``config.ini`` file.
-.. index:: config.ini; sections main
-Section **main**
-database -- ``db``
-Database directory path. The path may be either absolute or relative
-to the directory containig this config file.
-templates -- ``html``
-Path to the HTML templates directory. The path may be either absolute
-or relative to the directory containing this config file.
-static_files -- default *blank*
-A list of space separated directory paths (or a single directory).
-These directories hold additional static files available via Web UI.
-These directories may contain sitewide images, CSS stylesheets etc. If
-a '-' is included, the list processing ends and the TEMPLATES
-directory is not searched after the specified directories.  If this
-option is not set, all static files are taken from the TEMPLATES
-directory.
-admin_email -- ``roundup-admin``
-Email address that roundup will complain to if it runs into trouble. If
-the email address doesn't contain an ``@`` part, the MAIL_DOMAIN defined
-below is used.
-dispatcher_email -- ``roundup-admin``
-The 'dispatcher' is a role that can get notified of new items to the
-database. It is used by the ERROR_MESSAGES_TO config setting. If the
-email address doesn't contain an ``@`` part, the MAIL_DOMAIN defined
-below is used.
-email_from_tag -- default *blank*
-Additional text to include in the "name" part of the From: address used
-in nosy messages. If the sending user is "Foo Bar", the From: line
-is usually: ``"Foo Bar" <issue_tracker@tracker.example>``
-the EMAIL_FROM_TAG goes inside the "Foo Bar" quotes like so:
-``"Foo Bar EMAIL_FROM_TAG" <issue_tracker@tracker.example>``
-new_web_user_roles -- ``User``
-Roles that a user gets when they register with Web User Interface.
-This is a comma-separated list of role names (e.g. ``Admin,User``).
-new_email_user_roles -- ``User``
-Roles that a user gets when they register with Email Gateway.
-This is a comma-separated string of role names (e.g. ``Admin,User``).
-error_messages_to -- ``user``
-Send error message emails to the ``dispatcher``, ``user``, or ``both``?
-The dispatcher is configured using the DISPATCHER_EMAIL setting.
-Allowed values: ``dispatcher``, ``user``, or ``both``
-html_version -- ``html4``
-HTML version to generate. The templates are ``html4`` by default.
-If you wish to make them xhtml, then you'll need to change this
-var to ``xhtml`` too so all auto-generated HTML is compliant.
-Allowed values: ``html4``, ``xhtml``
-timezone -- ``0``
-Numeric timezone offset used when users do not choose their own
-in their settings.
-instant_registration -- ``yes``
-Register new users instantly, or require confirmation via
-email?
-Allowed values: ``yes``, ``no``
-email_registration_confirmation -- ``yes``
-Offer registration confirmation by email or only through the web?
-Allowed values: ``yes``, ``no``
-indexer_stopwords -- default *blank*
-Additional stop-words for the full-text indexer specific to
-your tracker. See the indexer source for the default list of
-stop-words (e.g. ``A,AND,ARE,AS,AT,BE,BUT,BY, ...``).
-umask -- ``02``
-Defines the file creation mode mask.
-csv_field_size -- ``131072``
-Maximum size of a csv-field during import. Roundups export
-format is a csv (comma separated values) variant. The csv
-reader has a limit on the size of individual fields
-starting with python 2.5. Set this to a higher value if you
-get the error 'Error: field larger than field limit' during
-import.
-.. index:: config.ini; sections tracker
-Section **tracker**
-name -- ``Roundup issue tracker``
-A descriptive name for your roundup instance.
-web -- ``http://host.example/demo/``
-The web address that the tracker is viewable at.
-This will be included in information sent to users of the tracker.
-The URL MUST include the cgi-bin part or anything else
-that is required to get to the home page of the tracker.
-You MUST include a trailing '/' in the URL.
-email -- ``issue_tracker``
-Email address that mail to roundup should go to.
-language -- default *blank*
-Default locale name for this tracker. If this option is not set, the
-language is determined by the environment variable LANGUAGE, LC_ALL,
-LC_MESSAGES, or LANG, in that order of preference.
-.. index:: config.ini; sections web
-Section **web**
-allow_html_file -- ``no``
-Setting this option enables Roundup to serve uploaded HTML
-file content *as HTML*. This is a potential security risk
-and is therefore disabled by default. Set to 'yes' if you
-trust *all* users uploading content to your tracker.
-http_auth -- ``yes``
-Whether to use HTTP Basic Authentication, if present.
-Roundup will use either the REMOTE_USER or HTTP_AUTHORIZATION
-variables supplied by your web server (in that order).
-Set this option to 'no' if you do not wish to use HTTP Basic
-Authentication in your web interface.
-use_browser_language -- ``yes``
-Whether to use HTTP Accept-Language, if present.
-Browsers send a language-region preference list.
-It's usually set in the client's browser or in their
-Operating System.
-Set this option to 'no' if you want to ignore it.
-debug -- ``no``
-Setting this option makes Roundup display error tracebacks
-in the user's browser rather than emailing them to the
-tracker admin."),
-.. index:: config.ini; sections rdbms
-single: config.ini; database settings
-Section **rdbms**
-Settings in this section are used to set the backend and configure
-addition settings needed by RDBMs like SQLite, Postgresql and
-MySQL backends.
-.. index::
-single: postgres; select backend in config.ini
-single: mysql; select backend in config.ini
-single: sqlite; select backend in config.ini
-single: anydbm; select backend in config.ini
-see: database; postgres
-see: database; mysql
-see: database; sqlite
-see: database; anydbm
-backend -- set to value by init
-The database backend such as anydbm, sqlite, mysql or postgres.
-name -- ``roundup``
-Name of the database to use.
-host -- ``localhost``
-Database server host.
-port -- default *blank*
-TCP port number of the database server. Postgresql usually resides on
-port 5432 (if any), for MySQL default port number is 3306. Leave this
-option empty to use backend default.
-user -- ``roundup``
-Database user name that Roundup should use.
-password -- ``roundup``
-Database user password.
-read_default_file -- ``~/.my.cnf``
-Name of the MySQL defaults file. Only used in MySQL connections.
-read_default_group -- ``roundup``
-Name of the group to use in the MySQL defaults file. Only used in
-MySQL connections.
-.. index::
-single: sqlite; lock timeout
-sqlite_timeout -- ``30``
-Number of seconds to wait when the SQLite database is locked.
-Used only for SQLite.
-cache_size -- `100`
-Size of the node cache (in elements) used to keep most recently used
-data in memory.
-.. index:: config.ini; sections logging
-see: logging; config.ini, sections logging
-Section **logging**
-config -- default *blank*
-Path to configuration file for standard Python logging module. If this
-option is set, logging configuration is loaded from specified file;
-options 'filename' and 'level' in this section are ignored. The path may
-be either absolute or relative to the directory containig this config file.
-filename -- default *blank*
-Log file name for minimal logging facility built into Roundup.  If no file
-name specified, log messages are written on stderr. If above 'config'
-option is set, this option has no effect. The path may be either absolute
-or relative to the directory containig this config file.
-level -- ``ERROR``
-Minimal severity level of messages written to log file. If above 'config'
-option is set, this option has no effect.
-Allowed values: ``DEBUG``, ``INFO``, ``WARNING``, ``ERROR``
-.. index:: config.ini; sections mail
-Section **mail**
-Outgoing email options. Used for nosy messages, password reset and
-registration approval requests.
-domain -- ``localhost``
-Domain name used for email addresses.
-host -- default *blank*
-SMTP mail host that roundup will use to send mail
-username -- default *blank*
-SMTP login name. Set this if your mail host requires authenticated access.
-If username is not empty, password (below) MUST be set!
-password -- default *blank*
-SMTP login password.
-Set this if your mail host requires authenticated access.
-port -- default *25*
-SMTP port on mail host.
-Set this if your mail host runs on a different port.
-local_hostname -- default *blank*
-The fully qualified domain name (FQDN) to use during SMTP sessions. If left
-blank, the underlying SMTP library will attempt to detect your FQDN. If your
-mail host requires something specific, specify the FQDN to use.
-tls -- ``no``
-If your SMTP mail host provides or requires TLS (Transport Layer Security)
-then you may set this option to 'yes'.
-Allowed values: ``yes``, ``no``
-tls_keyfile -- default *blank*
-If TLS is used, you may set this option to the name of a PEM formatted
-file that contains your private key. The path may be either absolute or
-relative to the directory containig this config file.
-tls_certfile -- default *blank*
-If TLS is used, you may set this option to the name of a PEM formatted
-certificate chain file. The path may be either absolute or relative
-to the directory containig this config file.
-charset -- utf-8
-Character set to encode email headers with. We use utf-8 by default, as
-it's the most flexible. Some mail readers (eg. Eudora) can't cope with
-that, so you might need to specify a more limited character set
-(eg. iso-8859-1).
-debug -- default *blank*
-Setting this option makes Roundup to write all outgoing email messages
-to this file *instead* of sending them. This option has the same effect
-as environment variable SENDMAILDEBUG. Environment variable takes
-precedence. The path may be either absolute or relative to the directory
-containig this config file.
-add_authorinfo -- ``yes``
-Add a line with author information at top of all messages send by
-roundup.
-add_authoremail -- ``yes``
-Add the mail address of the author to the author information at the
-top of all messages.  If this is false but add_authorinfo is true,
-only the name of the actor is added which protects the mail address
-of the actor from being exposed at mail archives, etc.
-.. index:: config.ini; sections mailgw
-single: mailgw; config
-see: mail gateway; mailgw
-Section **mailgw**
-Roundup Mail Gateway options
-keep_quoted_text -- ``yes``
-Keep email citations when accepting messages. Setting this to ``no`` strips
-out "quoted" text from the message. Signatures are also stripped.
-Allowed values: ``yes``, ``no``
-leave_body_unchanged -- ``no``
-Preserve the email body as is - that is, keep the citations *and*
-signatures.
-Allowed values: ``yes``, ``no``
-default_class -- ``issue``
-Default class to use in the mailgw if one isn't supplied in email subjects.
-To disable, leave the value blank.
-language -- default *blank*
-Default locale name for the tracker mail gateway.  If this option is
-not set, mail gateway will use the language of the tracker instance.
-subject_prefix_parsing -- ``strict``
-Controls the parsing of the [prefix] on subject lines in incoming emails.
-``strict`` will return an error to the sender if the [prefix] is not
-recognised. ``loose`` will attempt to parse the [prefix] but just
-pass it through as part of the issue title if not recognised. ``none``
-will always pass any [prefix] through as part of the issue title.
-subject_suffix_parsing -- ``strict``
-Controls the parsing of the [suffix] on subject lines in incoming emails.
-``strict`` will return an error to the sender if the [suffix] is not
-recognised. ``loose`` will attempt to parse the [suffix] but just
-pass it through as part of the issue title if not recognised. ``none``
-will always pass any [suffix] through as part of the issue title.
-subject_suffix_delimiters -- ``[]``
-Defines the brackets used for delimiting the commands suffix in a subject
-line.
-subject_content_match -- ``always``
-Controls matching of the incoming email subject line against issue titles
-in the case where there is no designator [prefix]. ``never`` turns off
-matching. ``creation + interval`` or ``activity + interval`` will match
-an issue for the interval after the issue's creation or last activity.
-The interval is a standard Roundup interval.
-subject_updates_title -- ``yes``
-Update issue title if incoming subject of email is different.
-Setting this to ``no`` will ignore the title part of
-the subject of incoming email messages.
-refwd_re -- ``(\s*\W?\s*(fw|fwd|re|aw|sv|ang)\W)+``
-Regular expression matching a single reply or forward prefix
-prepended by the mailer. This is explicitly stripped from the
-subject during parsing.  Value is Python Regular Expression
-(UTF8-encoded).
-origmsg_re -- `` ^[>|\s]*-----\s?Original Message\s?-----$``
-Regular expression matching start of an original message if quoted
-in the body.  Value is Python Regular Expression (UTF8-encoded).
-sign_re -- ``^[>|\s]*-- ?$``
-Regular expression matching the start of a signature in the message
-body.  Value is Python Regular Expression (UTF8-encoded).
-eol_re -- ``[\r\n]+``
-Regular expression matching end of line.  Value is Python Regular
-Expression (UTF8-encoded).
-blankline_re -- ``[\r\n]+\s*[\r\n]+``
-Regular expression matching a blank line.  Value is Python Regular
-Expression (UTF8-encoded).
-ignore_alternatives -- ``no``
-When parsing incoming mails, roundup uses the first
-text/plain part it finds. If this part is inside a
-multipart/alternative, and this option is set, all other
-parts of the multipart/alternative are ignored. The default
-is to keep all parts and attach them to the issue.
-.. index:: config.ini; sections php
-Section **pgp**
-OpenPGP mail processing options
-enable -- ``no``
-Enable PGP processing. Requires gpg.
-roles -- default *blank*
-If specified, a comma-separated list of roles to perform PGP
-processing on. If not specified, it happens for all users.
-homedir -- default *blank*
-Location of PGP directory. Defaults to $HOME/.gnupg if not
-specified.
-.. index:: config.ini; sections nosy
-Section **nosy**
-Nosy messages sending
-messages_to_author -- ``no``
-Send nosy messages to the author of the message.
-If ``yes`` is used, then messages are sent to the author
-even if not on the nosy list, same for ``new`` (but only for new messages).
-When set to ``nosy``, the nosy list controls sending messages to the author.
-Allowed values: ``yes``, ``no``, ``new``, ``nosy``
-signature_position -- ``bottom``
-Where to place the email signature.
-Allowed values: ``top``, ``bottom``, ``none``
-add_author -- ``new``
-Does the author of a message get placed on the nosy list automatically?
-If ``new`` is used, then the author will only be added when a message
-creates a new issue. If ``yes``, then the author will be added on
-followups too. If ``no``, they're never added to the nosy.
-Allowed values: ``yes``, ``no``, ``new``
-add_recipients -- ``new``
-Do the recipients (``To:``, ``Cc:``) of a message get placed on the nosy
-list?  If ``new`` is used, then the recipients will only be added when a
-message creates a new issue. If ``yes``, then the recipients will be added
-on followups too. If ``no``, they're never added to the nosy.
-Allowed values: ``yes``, ``no``, ``new``
-email_sending -- ``single``
-Controls the email sending from the nosy reactor. If ``multiple`` then
-a separate email is sent to each recipient. If ``single`` then a single
-email is sent with each recipient as a CC address.
-max_attachment_size -- ``2147483647``
-Attachments larger than the given number of bytes won't be attached
-to nosy mails. They will be replaced by a link to the tracker's
-download page for the file.
-.. index:: single: roundup-admin; config.ini update
-single: roundup-admin; config.ini create
-single: config.ini; create
-single: config.ini; update
-You may generate a new default config file using the ``roundup-admin
-genconfig`` command. You can generate a new config file merging in
-existing settings using the ``roundup-admin updateconfig`` command.
-Configuration variables may be referred to in lower or upper case. In code,
-variables not in the "main" section are referred to using their section and
-name, so "domain" in the section "mail" becomes MAIL_DOMAIN.
-.. index:: pair: configuration; extensions
-pair: configuration; detectors
-Extending the configuration file
---------------------------------
-You can't add new variables to the config.ini file in the tracker home but
-you can add two new config.ini files:
-- a config.ini in the ``extensions`` directory will be loaded and attached
-to the config variable as "ext".
-- a config.ini in the ``detectors`` directory will be loaded and attached
-to the config variable as "detectors".
-For example, the following in ``detectors/config.ini``::
-[main]
-qa_recipients = email@example.com
-is accessible as::
-db.config.detectors['QA_RECIPIENTS']
-Note that the name grouping applied to the main configuration file is
-applied to the extension config files, so if you instead have::
-[qa]
-recipients = email@example.com
-then the above ``db.config.detectors['QA_RECIPIENTS']`` will still work.
-Unlike values in the tracker's main ``config.ini``, the values defined
-in these config files are not validated. For example: a setting that
-is supposed to be an integer value (e.g. 4) could be the word
-"foo". If you are writing Python code that uses these settings, you
-should expect to handle invalid values.
-Also, incorrect values aren't discovered until the config setting is
-used. This can be long after the tracker is started and the error may
-not be seen in the logs.
-It is possible to validate these settings. Validation involves calling
-the ``update_options`` method on the configuration option. This can be
-done from the ``init()`` function in the Python files implementing
-extensions_ or detectors_.
-As an example, adding the following to an extension::
-from roundup.configuration import SecretMandatoryOption
-def init(instance):
-instance.config.ext.update_option('RECAPTCHA_SECRET',
-SecretMandatoryOption,description="Secret securing reCaptcha.")
-similarly for a detector::
-from roundup.configuration import MailAddressOption
-def init(db):
-try:
-db.config.detectors.update_option('QA_RECIPIENTS',
-MailAddressOption,
-	       description="Email used for QA comment followup.")
-except KeyError:
-# COMMENT_EMAIL setting is not found, but it's optional
-	   # so continue
-pass
-will allow reading the secret from a file or append the tracker domain
-to an email address if it does not have a domain.
-Running ``roundup-admin -i tracker_home display user1`` will validate
-the settings for both config.ini`s. Otherwise detector options are not
-validated until the first request to the web interface (or email
-gateway).
-There are 4 arguments for ``update_option``:
-1. config setting name - string (positional, mandatory)
-2. option type - Option derived class from configuration.py
-(positional, mandatory)
-3. default value - string (optional, named default)
-4. description - string (optional, named description)
-The first argument is the config setting name as described at the
-beginning of this section.
-The second argument is a class in the roundup.configuration module.
-There are a number of these classes: BooleanOption,
-IntegerNumberOption, RegExpOption.... Please see the configuration
-module for all Option validators and their descriptions. You can also
-define your own custom validator in `interfaces.py`_.
-The third and fourth arguments are strings and are optional. They are
-printed if there is an error and may help the user correct the problem.
-.. index:: ! schema
-Tracker Schema
-==============
-.. note::
-if you modify the schema, you'll most likely need to edit the
-`web interface`_ HTML template files and `detectors`_ to reflect
-your changes.
-A tracker schema defines what data is stored in the tracker's database.
-Schemas are defined using Python code in the ``schema.py`` module of your
-tracker.
-The ``schema.py`` and ``initial_data.py`` modules
--------------------------------------------------
-The schema.py module is used to define what your tracker looks like
-on the inside, the schema of the tracker. It defines the Classes
-and properties on each class. It also defines the security for
-those Classes. The next few sections describe how schemas work
-and what you can do with them.
-The initial_data.py module sets up the initial state of your
-tracker. It’s called exactly once - by the ``roundup-admin initialise``
-command. See the start of the section on database content for more
-info about how this works.
-.. index:: schema; classic - description of
-The "classic" schema
---------------------
-The "classic" schema looks like this (see section `setkey(property)`_
-below for the meaning of ``'setkey'`` -- you may also want to look into
-the sections `setlabelprop(property)`_ and `setorderprop(property)`_ for
-specifying (default) labelling and ordering of classes.)::
-pri = Class(db, "priority", name=String(), order=String())
-pri.setkey("name")
-stat = Class(db, "status", name=String(), order=String())
-stat.setkey("name")
-keyword = Class(db, "keyword", name=String())
-keyword.setkey("name")
-user = Class(db, "user", username=String(), organisation=String(),
-password=String(), address=String(), realname=String(),
-phone=String(), alternate_addresses=String(),
-queries=Multilink('query'), roles=String(), timezone=String())
-user.setkey("username")
-msg = FileClass(db, "msg", author=Link("user"), summary=String(),
-date=Date(), recipients=Multilink("user"),
-files=Multilink("file"), messageid=String(), inreplyto=String())
-file = FileClass(db, "file", name=String())
-issue = IssueClass(db, "issue", keyword=Multilink("keyword"),
-status=Link("status"), assignedto=Link("user"),
-priority=Link("priority"))
-issue.setkey('title')
-.. index:: schema; allowed changes
-What you can't do to the schema
--------------------------------
-You must never:
-**Remove the users class**
-This class is the only *required* class in Roundup.
-**Remove the "username", "address", "password" or "realname" user properties**
-Various parts of Roundup require these properties. Don't remove them.
-**Change the type of a property**
-Property types must *never* be changed - the database simply doesn't take
-this kind of action into account. Note that you can't just remove a
-property and re-add it as a new type either. If you wanted to make the
-assignedto property a Multilink, you'd need to create a new property
-assignedto_list and remove the old assignedto property.
-What you can do to the schema
------------------------------
-Your schema may be changed at any time before or after the tracker has been
-initialised (or used). You may:
-**Add new properties to classes, or add whole new classes**
-This is painless and easy to do - there are generally no repercussions
-from adding new information to a tracker's schema.
-**Remove properties**
-Removing properties is a little more tricky - you need to make sure that
-the property is no longer used in the `web interface`_ *or* by the
-detectors_.
-Classes and Properties - creating a new information store
----------------------------------------------------------
-In the tracker above, we've defined 7 classes of information:
-priority
-Defines the possible levels of urgency for issues.
-status
-Defines the possible states of processing the issue may be in.
-keyword
-Initially empty, will hold keywords useful for searching issues.
-user
-Initially holding the "admin" user, will eventually have an entry
-for all users using roundup.
-msg
-Initially empty, will hold all e-mail messages sent to or
-generated by roundup.
-file
-Initially empty, will hold all files attached to issues.
-issue
-Initially empty, this is where the issue information is stored.
-We define the "priority" and "status" classes to allow two things:
-1. reduction in the amount of information stored on the issue
-2. more powerful, accurate searching of issues by priority and status
-By only requiring a link on the issue (which is stored as a single
-number) we reduce the chance that someone mis-types a priority or
-status - or simply makes a new one up.
-Class and Items
-~~~~~~~~~~~~~~~
-A Class defines a particular class (or type) of data that will be stored
-in the database. A class comprises one or more properties, which gives
-the information about the class items.
-The actual data entered into the database, using ``class.create()``, are
-called items. They have a special immutable property called ``'id'``. We
-sometimes refer to this as the *itemid*.
-.. index:: schema; property types
-Properties
-~~~~~~~~~~
-A Class is comprised of one or more properties of the following types:
-String
-properties are for storing arbitrary-length strings.
-Password
-properties are for storing encoded arbitrary-length strings.
-The default encoding is defined on the ``roundup.password.Password``
-class.
-Computed
-properties invoke a python function. The return value of the
-function is the value of the property. Unlike other properties,
-the property is read only and can not be changed. Use cases:
-ask a remote interface for a value (e.g. retrieve user's office
-location from ldap, query state of a related ticket from
-another roundup instance). It can be used to compute a value
-(e.g. count the number of messages for an issue). The type
-returned by the function is the type of the value. (Note it is
-coerced to a string when displayed in the html interface.) At
-this time it's a partial implementation. It can be
-displayed/queried only. It can not be searched or used for
-sorting or grouping as it does not exist in the back end
-database.
-Date
-properties store date-and-time stamps. Their values are Timestamp
-objects.
-Interval
-properties store time periods rather than absolute dates. For
-example 2 hours.
-Integer
-properties store integer values. (Number can store real/float values.)
-Number
-properties store numeric values. There is an option to use
-double-precision floating point numbers.
-Boolean
-properties store on/off, yes/no, true/false values.
-Link
-properties refers to a single other item selected from a
-specified class. The class is part of the property; the value is an
-integer, the id of the chosen item.
-Multilink
-properties refer to possibly many items in a specified
-class. The value is a list of integers.
-Properties can have additional attributes to change the default
-behaviour:
-.. index:: triple: schema; property attributes; required
-triple: schema; property attributes; default_value
-triple: schema; property attributes; quiet
-* All properties support the following attributes:
-- ``required``: see `design documentation`_. Adds the property to
-the list returned by calling get_required_props for the class.
-- ``default_value``: see `design documentation`_ Sets the default
-value if the property is not set.
-- ``quiet``: see `design documentation`_. Suppresses user visible
-to changes to this property. The property change is not reported:
-- in the change feedback/confirmation message in the web
-interface
-- the property change section of the nosy email
-- the web history at the bottom of an item's page
-This can be used to store state of the user interface (e.g. the
-names of elements that are collapsed or hidden from the
-user). Making properties that are updated as an indirect result of
-a user's change (e.g. updating a blockers property, counting
-number of times an issue was reopened or reassigned etc.) should
-not be displayed to the user as they can be confusing.
-.. index:: triple: schema; property attributes; indexme
-* String properties can have an ``indexme`` attribute that defines if the
-property should be part of the full text index. The default is 'no' but this
-can be set to 'yes' to allow a property's contents to be in the full
-text index.
-.. index:: triple: schema; property attributes; use_double
-* Number properties can have a ``use_double`` attribute that, when set
-to ``True``, will use double precision floating point in the database.
-* Link and Multilink properties can have several attributes:
-.. index:: triple: schema; property attributes; do_journal
-- ``do_journal``: By default, every change of a link property is
-recorded in the item being linked to (or being unlinked). A typical
-use-case for setting ``do_journal='no'`` would be to turn off
-journalling of nosy list, message author and message recipient link
-and unlink events to prevent the journal from clogged with these
-events.
-.. index:: triple: schema; property attributes; try_id_parsing
-- ``try_id_parsing`` is turned on by default. If entering a number
-into a Link or Multilink field, roundup interprets this number as an
-ID of the item to link to. Sometimes items can have numeric names
-(like, e.g., product codes). For these roundup needs to match the
-numeric name and should never match an ID. In this case you can set
-``try_id_parsing='no'``.
-.. index:: triple: schema; property attributes; rev_multilink
-- The ``rev_multilink`` option takes a property name to be inserted
-into the linked-to class. This property is a Multilink property that
-links back to the current class. The new Multilink is read-only (it
-is automatically modified if the Link or Multilink property defining
-it is modified). The new property can be used in normal searches
-using the "filter" method of the Class. This means it can be used
-like other Multilink properties when searching (in an index
-template) or via the REST and XMLRPC APIs.
-As a example, suppose you want to group multiple issues into a
-super issue. Each issue can be part of only one super issue. It is
-inefficient to find all of the issues that are part of the
-super issue by searching through all issues in the system looking
-at the part_of link property. To make this more efficient, you
-can declare an issue's part_of property as::
-issue = IssueClass(db, "issue",
-...
-		 part_of = Link("issue", rev_multilink="components"),
-		 ... )
-This automatically creates the ``components`` multilink on the issue
-class. The ``components`` multilink is never explicitly declared in
-the issue class, but it has the same effect as though you had
-declared the class as::
-issue = IssueClass(db, "issue",
-...
-		 part_of = Link("issue"),
-		 components = Multilink("issue"),
-		 ... )
-Then wrote a detector to update the components property on the
-corresponding issue. Writing this detector can be tricky. There is
-one other difference, you can not explicitly set/modify the
-``components`` multilink.
-The effect of setting ``part_of = 3456`` on issue1234
-automatically adds "1234" to the ``components`` property on
-issue3456. You can search the ``components`` multilink just like a
-regular multilink, but you can't explicitly assign to it.
-Another difference of reverse multilinks to normal multilinks
-is that when a linked node is retired, the node vanishes from the
-multilink, e.g. in the example above, if an issue with ``part_of``
-set to another issue is retired this issue vanishes from the
-``components`` multilink of the other issue.
-You can also link between different classes. So you can modify
-the issue definition to include::
-issue = IssueClass(db, "issue",
-...
-		 assigned_to = Link("user", rev_multilink="responsibleFor"),
-		 ... )
-This makes it easy to list all issues that the user is responsible
+<tr>
-for (aka assigned_to).
+<th>Due Date</th>
+<td tal:content="structure python:context.due_date.field(format='%Y-%m-%d')" />
-.. index:: triple: schema; property attributes; msg_header_property
+</tr>
-- The ``msg_header_property`` is used by the mail gateway when sending
+3. Add the property to the ``issue.index.html`` page::
-out messages. When a link or multilink property of an issue changes,
-roundup creates email headers of the form::
+(in the heading row)
+<th tal:condition="request/show/due_date">Due Date</th>
-X-Roundup-issue-prop: value
+(in the data row)
+<td tal:condition="request/show/due_date"
-where ``value`` is the ``name`` property for the linked item(s).
+tal:content="i/due_date" />
-For example, if you have a multilink for attached_files in your
-issue, you will see a header::
+If you want format control of the display of the due date you can
+enter the following in the data row to show only the actual due date::
-X-Roundup-issue-attached_files: MySpecialFile.doc, HisResume.txt
+<td tal:condition="request/show/due_date"
-when the class for attached files is defined as::
+tal:content="python:i.due_date.pretty('%Y-%m-%d')">&nbsp;</td>
-file = FileClass(db, "file", name=String())
+4. Add the property to the ``issue.search.html`` page::
-``MySpecialFile.doc`` is the name for the file object.
+<tr tal:define="name string:due_date">
+<th i18n:translate="">Due Date:</th>
-If you have an ``assigned_to`` property in your issue class that
+<td metal:use-macro="search_input"></td>
-links to the user class and you want to add a header::
+<td metal:use-macro="column_input"></td>
+<td metal:use-macro="sort_input"></td>
-X-Roundup-issue-assigned_to: ...
+<td metal:use-macro="group_input"></td>
+</tr>
-so that the mail recipients can filter emails where
-``X-Roundup-issue-assigned_to: name`` that contains their
+5. If you wish for the due date to appear in the standard views listed
-username. The user class is defined as::
+in the sidebar of the web interface then you'll need to add "due_date"
+to the columns and columns_showall lists in your ``page.html``::
-user = Class(db, "user",
+columns string:id,activity,due_date,title,creator,status;
+columns_showall string:id,activity,due_date,title,creator,assignedto,status;
+Adding a new Computed field to the schema
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Computed properties are a bit different from other properties. They do
+not actually change the database. Computed fields are not contained in
+the database and can not be searched or used for sorting or
+grouping. (Patches to add this capability are welcome.)
+In this example we will add a count of the number of files attached to
+the issue. This could be done using an auditor (see below) to update
+an integer field called ``filecount``. But we will implement this in a
+different way.
+We have two changes to make:
+1. add a new python method to the hyperdb.Computed class. It will
+count the number of files attached to the issue. This method will
+be added in the (possibly new) interfaces.py file in the top level
+of the tracker directory.  (See interfaces.py above for more
+information.)
+2. add a new ``filecount`` property to the issue class calling the
+new function.
+A Computed method receives three arguments when called:
+1. the computed object (self)
+2. the id of the item in the class
+3. the database object
+To add the method to the Computed class, modify the trackers
+interfaces.py adding::
+import roundup.hyperdb as hyperdb
+def filecount(self, nodeid, db):
+return len(db.issue.get(nodeid, 'files'))
+setattr(hyperdb.Computed, 'filecount', filecount)
+Then add::
+filecount = Computed(Computed.filecount),
+to the existing IssueClass call.
+Now you can retrieve the value of the ``filecount`` property and it
+will be computed on the fly from the existing list of attached files.
+This example was done with the IssueClass, but you could add a
+Computed property to any class. E.G.::
+user = Class(db, "user",
 username=String(),
 password=Password(),
 address=String(),
 realname=String(),
 phone=String(),
+office=Computed(Computed.getOfficeFromLdap), # new prop
 organisation=String(),
 alternate_addresses=String(),
 queries=Multilink('query'),
-roles=String(),     # comma-separated string of Role names
+roles=String(),
 timezone=String())
-Because there is no ``name`` parameter for the user class, there
+where the method ``getOfficeFromLdap`` queries the local ldap server to
-will be no header. However setting::
+get the current office location information. The method will be called
+with the Computed instance, the user id and the database object.
-assigned_to=Link("user", msg_header_property="username")
+Adding a new constrained field to the classic schema
-will make the mail gateway generate an ``X-Roundup-issue-assigned_to``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-using the username property of the linked user.
+This example shows how to add a new constrained property (i.e. a
-Assume assigned_to for an issue is linked to the user with
+selection of distinct values) to your tracker.
-username=joe_user, setting::
-msg_header_property="username"
+Introduction
+::::::::::::
-for the assigned_to property will generated message headers of the
-form::
+To make the classic schema of Roundup useful as a TODO tracking system
+for a group of systems administrators, it needs an extra data field per
-X-Roundup-issue-assigned_to: joe_user
+issue: a category.
-for emails sent on issues where joe_user has been assigned to the issue.
+This would let sysadmins quickly list all TODOs in their particular area
+of interest without having to do complex queries, and without relying on
-If this property is set to the empty string "", it will prevent
+the spelling capabilities of other sysadmins (a losing proposition at
-the header from being generated on outgoing mail.
+best).
-.. index:: triple: schema; class property; creator
-triple: schema; class property; creation
+Adding a field to the database
-triple: schema; class property; actor
+::::::::::::::::::::::::::::::
-triple: schema; class property; activity
+This is the easiest part of the change. The category would just be a
-All Classes automatically have a number of properties by default:
+plain string, nothing fancy. To change what is in the database you need
+to add some lines to the ``schema.py`` file of your tracker instance.
-*creator*
+Under the comment::
-Link to the user that created the item.
-*creation*
+# add any additional database schema configuration here
-Date the item was created.
-*actor*
+add::
-Link to the user that last modified the item.
-*activity*
+category = Class(db, "category", name=String())
-Date the item was last modified.
+category.setkey("name")
+Here we are setting up a chunk of the database which we are calling
-.. index:: triple: schema; class property; content
+"category". It contains a string, which we are refering to as "name" for
-triple: schema; class property; type
+lack of a more imaginative title. (Since "name" is one of the properties
+that Roundup looks for on items if you do not set a key for them, it's
-FileClass
+probably a good idea to stick with it for new classes if at all
-~~~~~~~~~
+appropriate.) Then we are setting the key of this chunk of the database
+to be that "name". This is equivalent to an index for database types.
-FileClasses save their "content" attribute off in a separate file from
+This also means that there can only be one category with a given name.
-the rest of the database. This reduces the number of large entries in
-the database, which generally makes databases more efficient, and also
+Adding the above lines allows us to create categories, but they're not
-allows us to use command-line tools to operate on the files. They are
+tied to the issues that we are going to be creating. It's just a list of
-stored in the files sub-directory of the ``'db'`` directory in your
+categories off on its own, which isn't much use. We need to link it in
-tracker. FileClasses also have a "type" attribute to store the MIME
+with the issues. To do that, find the lines
-type of the file.
+in ``schema.py`` which set up the "issue" class, and then add a link to
+the category::
-Roundup by default considers the contents of the file immutable. This
-is to assist in maintaining an accurate record of correspondence.  The
+issue = IssueClass(db, "issue", ... ,
-distributed tracker templates do not enforce this. So if you have
+category=Multilink("category"), ... )
-access to the roundup tracker directory, you can edit the files (make
-sure to preserve mode, owner and group) to remove information (e.g. if
+The ``Multilink()`` means that each issue can have many categories. If
-somebody includes a password or you need to redact proprietary
+you were adding something with a one-to-one relationship to issues (such
-information). Obviously the journal for the message/file will not
+as the "assignedto" property), use ``Link()`` instead.
-report that the file has changed.
+That is all you need to do to change the schema. The rest of the effort
-Best practice is to remove offending material and leave a
+is fiddling around so you can actually use the new category.
-placeholder. E.G. replace a password with the text::
-[password has been deleted 2020-12-02 --myname]
+Populating the new category class
+:::::::::::::::::::::::::::::::::
-If you need to delete an entire file, replace the file contents with::
+If you haven't initialised the database with the
-[file contents deleted due to spam 202-10-21 --myname]
+"``roundup-admin initialise``" command, then you
+can add the following to the tracker ``initial_data.py``
-rather than deleting the file. If you actually delete the file roundup
+under the comment::
-will report an error to the user and email the administrator. If you
-empty the file, a user downloading the file using the direct URL
+# add any additional database creation steps here - but only if you
-(e.g. ``tracker/msg22``) may be confused and think something is broken
+# haven't initialised the database with the admin "initialise" command
-when they receive an empty file. Retiring a file/msg does not prevent
-access to the file using the direct URL. Retiring an item only removes
+Add::
-it when requesting a list of all items in the class. If you are
-replacing the contents, you probably want to change the content type
+category = db.getclass('category')
-of the file. E.G. from ``image/jpeg`` to ``text/plain``. You can do
+category.create(name="scipy")
-this easily through the web interface, or using the ``roundup-admin``
+category.create(name="chaco")
-command line interface.
+category.create(name="weave")
-You can also change the contents of a file or message using the REST
+.. index:: roundup-admin; create entries in class
-interface. Note that this will NOT result in an entry in the journal,
-so again it allows a silent change. To do this you need to make two
+If the database has already been initalised, then you need to use the
-rest requests. An example using curl is::
+``roundup-admin`` tool::
-$ curl -u demo:demo -s
+% roundup-admin -i <tracker home>
--H "X-requested-with: rest" \
+Roundup <version> ready for input.
--H "Referer:  https://tracker.example.com/demo/" \
+Type "help" for help.
--X GET \
+roundup> create category name=scipy
-https://tracker.example.com/demo/rest/data/file/30/content
+1
-{
+roundup> create category name=chaco
-"data": {
+2
-"id": "30",
+roundup> create category name=weave
-"type": "<class 'str'>",
+3
-"link": "https://tracker.example.com/demo/rest/data/file/30/conten
+roundup> exit...
-t",
+There are unsaved changes. Commit them (y/N)? y
-"data": "hello3",
-"@etag": "\"3f2f8063dbce5b6bd43567e6f4f3c671\""
-}
+Setting up security on the new objects
-}
+::::::::::::::::::::::::::::::::::::::
-using the etag, overwrite the content with::
+By default only the admin user can look at and change objects. This
+doesn't suit us, as we want any user to be able to create new categories
-$ curl -u demo:demo -s
+as required, and obviously everyone needs to be able to view the
--H "X-requested-with: rest" \
+categories of issues for it to be useful.
--H "Referer:  https://tracker.example.com/demo/" \
--H 'If-Match: "3f2f8063dbce5b6bd43567e6f4f3c671"' \
+We therefore need to change the security of the category objects. This
--X PUT \
+is also done in ``schema.py``.
--F "data=@hello" \
-https://tracker.example.com/demo/rest/data/file/30/content
+There are currently two loops which set up permissions and then assign
+them to various roles. Simply add the new "category" to both lists::
-where ``hello`` is a file on local disk.
+# Assign the access and edit permissions for issue, file and message
-You can enforce immutability in your tracker by adding an auditor (see
+# to regular users now
-detectors_) for the file/msg class that rejects changes to the content
+for cl in 'issue', 'file', 'msg', 'category':
-property. The auditor could also add a journal entry so that a change
+p = db.security.getPermission('View', cl)
-via the roundup mechanism is reported.  Using a mixin (see:
+db.security.addPermissionToRole('User', 'View', cl)
-https://wiki.roundup-tracker.org/MixinClassFileClass) to augment the
+db.security.addPermissionToRole('User', 'Edit', cl)
-file class allows for other possibilities including signing the file, or
+db.security.addPermissionToRole('User', 'Create', cl)
-recording a checksum in the database and validating the file contents
-at the time it gets read. This allows detection of changes done on the
+These lines assign the "View" and "Edit" Permissions to the "User" role,
-filesystem outside of the roundup mechanism.
+so that normal users can view and edit "category" objects.
-.. index:: triple: schema; class property; messages
+This is all the work that needs to be done for the database. It will
-triple: schema; class property; files
+store categories, and let users view and edit them. Now on to the
-triple: schema; class property; nosy
+interface stuff.
-triple: schema; class property; superseder
-IssueClass
+Changing the web left hand frame
-~~~~~~~~~~
+::::::::::::::::::::::::::::::::
-IssueClasses automatically include the "messages", "files", "nosy", and
+We need to give the users the ability to create new categories, and the
-"superseder" properties.
+place to put the link to this functionality is in the left hand function
+bar, under the "Issues" area. The file that defines how this area looks
-The messages and files properties list the links to the messages and
+is ``html/page.html``, which is what we are going to be editing next.
-files related to the issue. The nosy property is a list of links to
-users who wish to be informed of changes to the issue - they get "CC'ed"
+If you look at this file you can see that it contains a lot of
-e-mails when messages are sent to or generated by the issue. The nosy
+"classblock" sections which are chunks of HTML that will be included or
-reactor (in the ``'detectors'`` directory) handles this action. The
+excluded in the output depending on whether the condition in the
-superseder link indicates an issue which has superseded this one.
+classblock is met. We are going to add the category code at the end of
+the classblock for the *issue* class::
-They also have the dynamically generated "creation", "activity" and
-"creator" properties.
+<p class="classblock"
+tal:condition="python:request.user.hasPermission('View', 'category')">
-The value of the "creation" property is the date when an item was
+<b>Categories</b><br>
-created, and the value of the "activity" property is the date when any
+<a tal:condition="python:request.user.hasPermission('Edit', 'category')"
-property on the item was last edited (equivalently, these are the dates
+href="category?@template=item">New Category<br></a>
-on the first and last records in the item's journal). The "creator"
+</p>
-property holds a link to the user that created the issue.
+The first two lines is the classblock definition, which sets up a
-.. index: triple: schema; class method; setkey
+condition that only users who have "View" permission for the "category"
+object will have this section included in their output. Next comes a
-setkey(property)
+plain "Categories" header in bold. Everyone who can view categories will
-~~~~~~~~~~~~~~~~
+get that.
-.. index:: roundup-admin; setting assignedto on an issue
+Next comes the link to the editing area of categories. This link will
+only appear if the condition - that the user has "Edit" permissions for
-Select a String property of the class to be the key property. The key
+the "category" objects - is matched. If they do have permission then
-property must be unique, and allows references to the items in the class
+they will get a link to another page which will let the user add new
-by the content of the key property. That is, we can refer to users by
+categories.
-their username: for example, let's say that there's an issue in roundup,
-issue 23. There's also a user, richard, who happens to be user 2. To
+Note that if you have permission to *view* but not to *edit* categories,
-assign an issue to him, we could do either of::
+then all you will see is a "Categories" header with nothing underneath
+it. This is obviously not very good interface design, but will do for
-roundup-admin set issue23 assignedto=2
+now. I just claim that it is so I can add more links in this section
+later on. However, to fix the problem you could change the condition in
-or::
+the classblock statement, so that only users with "Edit" permission
+would see the "Categories" stuff.
-roundup-admin set issue23 assignedto=richard
-Note, the same thing can be done in the web and e-mail interfaces.
+Setting up a page to edit categories
+::::::::::::::::::::::::::::::::::::
-.. index: triple: schema; class method; setlabelprop
+We defined code in the previous section which let users with the
-setlabelprop(property)
+appropriate permissions see a link to a page which would let them edit
-~~~~~~~~~~~~~~~~~~~~~~
+conditions. Now we have to write that page.
-Select a property of the class to be the label property. The label
+The link was for the *item* template of the *category* object. This
-property is used whereever an item should be uniquely identified, e.g.,
+translates into Roundup looking for a file called ``category.item.html``
-when displaying a link to an item. If setlabelprop is not specified for
+in the ``html`` tracker directory. This is the file that we are going to
-a class, the following values are tried for the label:
+write now.
-* the key of the class (see the `setkey(property)`_ section above)
+First, we add an info tag in a comment which doesn't affect the outcome
-* the "name" property
+of the code at all, but is useful for debugging. If you load a page in a
-* the "title" property
+browser and look at the page source, you can see which sections come
-* the first property from the sorted property name list
+from which files by looking for these comments::
-So in most cases you can get away without specifying setlabelprop
+<!-- category.item -->
-explicitly.
+Next we need to add in the METAL macro stuff so we get the normal page
-You should make sure that users have View access to this property or
+trappings::
-the id property for a class. If the property can not be viewed by a
-user, looping over items in the class (e.g. messages attached to an
+<tal:block metal:use-macro="templates/page/macros/icing">
-issue) will not work.
+<title metal:fill-slot="head_title">Category editing</title>
+<td class="page-header-top" metal:fill-slot="body_title">
-.. index: triple: schema; class method; setorderprop
+<h2>Category editing</h2>
+</td>
-setorderprop(property)
+<td class="content" metal:fill-slot="content">
-~~~~~~~~~~~~~~~~~~~~~~
+Next we need to setup up a standard HTML form, which is the whole
-Select a property of the class to be the order property. The order
+purpose of this file. We link to some handy javascript which sends the
-property is used whenever using a default sort order for the class,
+form through only once. This is to stop users hitting the send button
-e.g., when grouping or sorting class A by a link to class B in the user
+multiple times when they are impatient and thus having the form sent
-interface, the order property of class B is used for sorting.  If
+multiple times::
-setorderprop is not specified for a class, the following values are tried
-for the order property:
+<form method="POST" onSubmit="return submit_once()"
+enctype="multipart/form-data">
-* the property named "order"
-* the label property (see `setlabelprop(property)`_ above)
+Next we define some code which sets up the minimum list of fields that
+we require the user to enter. There will be only one field - "name" - so
-So in most cases you can get away without specifying setorderprop
+they better put something in it, otherwise the whole form is pointless::
-explicitly.
+<input type="hidden" name="@required" value="name">
-.. index: triple: schema; class method; create
+To get everything to line up properly we will put everything in a table,
-create(information)
+and put a nice big header on it so the user has an idea what is
-~~~~~~~~~~~~~~~~~~~
+happening::
-Create an item in the database. This is generally used to create items
+<table class="form">
-in the "definitional" classes like "priority" and "status".
+<tr><th class="header" colspan="2">Category</th></tr>
-.. index: schema; item ordering
+Next, we need the field into which the user is going to enter the new
+category. The ``context.name.field(size=60)`` bit tells Roundup to
-A note about ordering
+generate a normal HTML field of size 60, and the contents of that field
-~~~~~~~~~~~~~~~~~~~~~
+will be the "name" variable of the current context (namely "category").
+The upshot of this is that when the user types something in
-When we sort items in the hyperdb, we use one of a number of methods,
+to the form, a new category will be created with that name::
-depending on the properties being sorted on:
+<tr>
-1. If it's a String, Integer, Number, Date or Interval property, we
+<th>Name</th>
-just sort the scalar value of the property. Strings are sorted
+<td tal:content="structure python:context.name.field(size=60)">
-case-sensitively.
+name</td>
-2. If it's a Link property, we sort by either the linked item's "order"
+</tr>
-property (if it has one) or the linked item's "id".
-3. Mulitlinks sort similar to #2, but we start with the first Multilink
+Then a submit button so that the user can submit the new category::
-list item, and if they're the same, we sort by the second item, and
-so on.
+<tr>
+<td>&nbsp;</td>
-Note that if an "order" property is defined on a Class that is used for
+<td colspan="3" tal:content="structure context/submit">
-sorting, all items of that Class *must* have a value against the "order"
+submit button will go here
-property, or sorting will result in random ordering.
+</td>
+</tr>
-Examples of adding to your schema
+The ``context/submit`` bit generates the submit button but also
+generates the @action and @csrf hidden fields. The @action field is
+used to tell Roundup how to process the form. The @csrf field provides
+a unique single use token to defend against CSRF attacks. (More about
+anti-csrf measures can be found in ``upgrading.txt``.)
+Finally we finish off the tags we used at the start to do the METAL
+stuff::
+</td>
+</tal:block>
+So putting it all together, and closing the table and form we get::
+<!-- category.item -->
+<tal:block metal:use-macro="templates/page/macros/icing">
+<title metal:fill-slot="head_title">Category editing</title>
+<td class="page-header-top" metal:fill-slot="body_title">
+<h2>Category editing</h2>
+</td>
+<td class="content" metal:fill-slot="content">
+<form method="POST" onSubmit="return submit_once()"
+enctype="multipart/form-data">
+<table class="form">
+<tr><th class="header" colspan="2">Category</th></tr>
+<tr>
+<th>Name</th>
+<td tal:content="structure python:context.name.field(size=60)">
+name</td>
+</tr>
+<tr>
+<td>
+&nbsp;
+<input type="hidden" name="@required" value="name">
+</td>
+<td colspan="3" tal:content="structure context/submit">
+submit button will go here
+</td>
+</tr>
+</table>
+</form>
+</td>
+</tal:block>
+This is quite a lot to just ask the user one simple question, but there
+is a lot of setup for basically one line (the form line) to do its work.
+To add another field to "category" would involve one more line (well,
+maybe a few extra to get the formatting correct).
+Adding the category to the issue
+::::::::::::::::::::::::::::::::
+We now have the ability to create issues to our heart's content, but
+that is pointless unless we can assign categories to issues.  Just like
+the ``html/category.item.html`` file was used to define how to add a new
+category, the ``html/issue.item.html`` is used to define how a new issue
+is created.
+Just like ``category.issue.html``, this file defines a form which has a
+table to lay things out. It doesn't matter where in the table we add new
+stuff, it is entirely up to your sense of aesthetics::
+<th>Category</th>
+<td>
+<span tal:replace="structure context/category/field" />
+<span tal:replace="structure python:db.category.classhelp('name',
+property='category', width='200')" />
+</td>
+First, we define a nice header so that the user knows what the next
+section is, then the middle line does what we are most interested in.
+This ``context/category/field`` gets replaced by a field which contains
+the category in the current context (the current context being the new
+issue).
+The classhelp lines generate a link (labelled "list") to a popup window
+which contains the list of currently known categories.
+Searching on categories
+:::::::::::::::::::::::
+Now we can add categories, and create issues with categories. The next
+obvious thing that we would like to be able to do, would be to search
+for issues based on their category, so that, for example, anyone working
+on the web server could look at all issues in the category "Web".
+If you look for "Search Issues" in the ``html/page.html`` file, you will
+find that it looks something like
+``<a href="issue?@template=search">Search Issues</a>``. This shows us
+that when you click on "Search Issues" it will be looking for a
+``issue.search.html`` file to display. So that is the file that we will
+change.
+If you look at this file it should begin to seem familiar, although it
+does use some new macros. You can add the new category search code anywhere you
+like within that form::
+<tr tal:define="name string:category;
+db_klass string:category;
+db_content string:name;">
+<th>Priority:</th>
+<td metal:use-macro="search_select"></td>
+<td metal:use-macro="column_input"></td>
+<td metal:use-macro="sort_input"></td>
+<td metal:use-macro="group_input"></td>
+</tr>
+The definitions in the ``<tr>`` opening tag are used by the macros:
+- ``search_select`` expands to a drop-down box with all categories using
+``db_klass`` and ``db_content``.
+- ``column_input`` expands to a checkbox for selecting what columns
+should be displayed.
+- ``sort_input`` expands to a radio button for selecting what property
+should be sorted on.
+- ``group_input`` expands to a radio button for selecting what property
+should be grouped on.
+The category search code above would expand to the following::
+<tr>
+<th>Category:</th>
+<td>
+<select name="category">
+<option value="">don't care</option>
+<option value="">------------</option>
+<option value="1">scipy</option>
+<option value="2">chaco</option>
+<option value="3">weave</option>
+</select>
+</td>
+<td><input type="checkbox" name=":columns" value="category"></td>
+<td><input type="radio" name=":sort0" value="category"></td>
+<td><input type="radio" name=":group0" value="category"></td>
+</tr>
+Adding category to the default view
+:::::::::::::::::::::::::::::::::::
+We can now add categories, add issues with categories, and search for
+issues based on categories. This is everything that we need to do;
+however, there is some more icing that we would like. I think the
+category of an issue is important enough that it should be displayed by
+default when listing all the issues.
+Unfortunately, this is a bit less obvious than the previous steps. The
+code defining how the issues look is in ``html/issue.index.html``. This
+is a large table with a form down at the bottom for redisplaying and so
+forth.
+Firstly we need to add an appropriate header to the start of the table::
+<th tal:condition="request/show/category">Category</th>
+The *condition* part of this statement is to avoid displaying the
+Category column if the user has selected not to see it.
+The rest of the table is a loop which will go through every issue that
+matches the display criteria. The loop variable is "i" - which means
+that every issue gets assigned to "i" in turn.
+The new part of code to display the category will look like this::
+<td tal:condition="request/show/category"
+tal:content="i/category"></td>
+The condition is the same as above: only display the condition when the
+user hasn't asked for it to be hidden. The next part is to set the
+content of the cell to be the category part of "i" - the current issue.
+Finally we have to edit ``html/page.html`` again. This time, we need to
+tell it that when the user clicks on "Unassigned Issues" or "All Issues",
+the category column should be included in the resulting list. If you
+scroll down the page file, you can see the links with lots of options.
+The option that we are interested in is the ``:columns=`` one which
+tells Roundup which fields of the issue to display. Simply add
+"category" to that list and it all should work.
+Adding a time log to your issues
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+We want to log the dates and amount of time spent working on issues, and
+be able to give a summary of the total time spent on a particular issue.
+1. Add a new class to your tracker ``schema.py``::
+# storage for time logging
+timelog = Class(db, "timelog", period=Interval())
+Note that we automatically get the date of the time log entry
+creation through the standard property "creation".
+You will need to grant "Creation" permission to the users who are
+allowed to add timelog entries. You may do this with::
+db.security.addPermissionToRole('User', 'Create', 'timelog')
+db.security.addPermissionToRole('User', 'View', 'timelog')
+If users are also able to *edit* timelog entries, then also include::
+db.security.addPermissionToRole('User', 'Edit', 'timelog')
+.. index:: schema; example changes
+2. Link to the new class from your issue class (again, in
+``schema.py``)::
+issue = IssueClass(db, "issue",
+assignedto=Link("user"), keyword=Multilink("keyword"),
+priority=Link("priority"), status=Link("status"),
+times=Multilink("timelog"))
+the "times" property is the new link to the "timelog" class.
+3. We'll need to let people add in times to the issue, so in the web
+interface we'll have a new entry field. This is a special field
+because unlike the other fields in the ``issue.item`` template, it
+affects a different item (a timelog item) and not the template's
+item (an issue). We have a special syntax for form fields that affect
+items other than the template default item (see the cgi
+documentation on `special form variables
+<reference.html#special-form-variables>`_). In particular, we add a
+field to capture a new timelog item's period::
+<tr>
+<th>Time Log</th>
+<td colspan=3><input type="text" name="timelog-1@period" />
+(enter as '3y 1m 4d 2:40:02' or parts thereof)
+</td>
+</tr>
+and another hidden field that links that new timelog item (new
+because it's marked as having id "-1") to the issue item. It looks
+like this::
+<input type="hidden" name="@link@times" value="timelog-1" />
+On submission, the "-1" timelog item will be created and assigned a
+real item id. The "times" property of the issue will have the new id
+added to it.
+The full entry will now look like this::
+<tr>
+<th>Time Log</th>
+<td colspan=3><input type="text" name="timelog-1@period" />
+(enter as '3y 1m 4d 2:40:02' or parts thereof)
+<input type="hidden" name="@link@times" value="timelog-1" />
+</td>
+</tr>
+.. _adding-a-time-log-to-your-issues-4:
+4. We want to display a total of the timelog times that have been
+accumulated for an issue. To do this, we'll need to actually write
+some Python code, since it's beyond the scope of PageTemplates to
+perform such calculations. We do this by adding a module ``timespent.py``
+to the ``extensions`` directory in our tracker. The contents of this
+file is as follows::
+from roundup import date
+def totalTimeSpent(times):
+''' Call me with a list of timelog items (which have an
+Interval "period" property)
+'''
+total = date.Interval('0d')
+for time in times:
+total += time.period._value
+return total
+def init(instance):
+instance.registerUtil('totalTimeSpent', totalTimeSpent)
+We will now be able to access the ``totalTimeSpent`` function via the
+``utils`` variable in our templates, as shown in the next step.
+5. Display the timelog for an issue::
+<table class="otherinfo" tal:condition="context/times">
+<tr><th colspan="3" class="header">Time Log
+<tal:block
+tal:replace="python:utils.totalTimeSpent(context.times)" />
+</th></tr>
+<tr><th>Date</th><th>Period</th><th>Logged By</th></tr>
+<tr tal:repeat="time context/times">
+<td tal:content="time/creation"></td>
+<td tal:content="time/period"></td>
+<td tal:content="time/creator"></td>
+</tr>
+</table>
+I put this just above the Messages log in my issue display. Note our
+use of the ``totalTimeSpent`` method which will total up the times
+for the issue and return a new Interval. That will be automatically
+displayed in the template as text like "+ 1y 2:40" (1 year, 2 hours
+and 40 minutes).
+6. If you're using a persistent web server - ``roundup-server`` or
+``mod_wsgi`` for example - then you'll need to restart that to pick up
+the code changes. When that's done, you'll be able to use the new
+time logging interface.
+An extension of this modification attaches the timelog entries to any
+change message entered at the time of the timelog entry:
+A. Add a link to the timelog to the msg class in ``schema.py``:
+msg = FileClass(db, "msg",
+author=Link("user", do_journal='no'),
+recipients=Multilink("user", do_journal='no'),
+date=Date(),
+summary=String(),
+files=Multilink("file"),
+messageid=String(),
+inreplyto=String(),
+times=Multilink("timelog"))
+B. Add a new hidden field that links that new timelog item (new
+because it's marked as having id "-1") to the new message.
+The link is placed in ``issue.item.html`` in the same section that
+handles the timelog entry.
+It looks like this after this addition::
+<tr>
+<th>Time Log</th>
+<td colspan=3><input type="text" name="timelog-1@period" />
+(enter as '3y 1m 4d 2:40:02' or parts thereof)
+<input type="hidden" name="@link@times" value="timelog-1" />
+<input type="hidden" name="msg-1@link@times" value="timelog-1" />
+</td>
+</tr>
+The "times" property of the message will have the new id added to it.
+C. Add the timelog listing from step 5. to the ``msg.item.html`` template
+so that the timelog entry appears on the message view page. Note that
+the call to totalTimeSpent is not used here since there will only be one
+single timelog entry for each message.
+I placed it after the Date entry like this::
+<tr>
+<th i18n:translate="">Date:</th>
+<td tal:content="context/date"></td>
+</tr>
+</table>
+<table class="otherinfo" tal:condition="context/times">
+<tr><th colspan="3" class="header">Time Log</th></tr>
+<tr><th>Date</th><th>Period</th><th>Logged By</th></tr>
+<tr tal:repeat="time context/times">
+<td tal:content="time/creation"></td>
+<td tal:content="time/period"></td>
+<td tal:content="time/creator"></td>
+</tr>
+</table>
+<table class="messages">
+Tracking different types of issues
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Sometimes you will want to track different types of issues - developer,
+customer support, systems, sales leads, etc. A single Roundup tracker is
+able to support multiple types of issues. This example demonstrates adding
+a system support issue class to a tracker.
+1. Figure out what information you're going to want to capture. OK, so
+this is obvious, but sometimes it's better to actually sit down for a
+while and think about the schema you're going to implement.
+2. Add the new issue class to your tracker's ``schema.py``. Just after the
+"issue" class definition, add::
+# list our systems
+system = Class(db, "system", name=String(), order=Number())
+system.setkey("name")
+# store issues related to those systems
+support = IssueClass(db, "support",
+assignedto=Link("user"), keyword=Multilink("keyword"),
+status=Link("status"), deadline=Date(),
+affects=Multilink("system"))
+3. Copy the existing ``issue.*`` (item, search and index) templates in the
+tracker's ``html`` to ``support.*``. Edit them so they use the properties
+defined in the ``support`` class. Be sure to check for hidden form
+variables like "required" to make sure they have the correct set of
+required properties.
+4. Edit the modules in the ``detectors``, adding lines to their ``init``
+functions where appropriate. Look for ``audit`` and ``react`` registrations
+on the ``issue`` class, and duplicate them for ``support``.
+5. Create a new sidebar box for the new support class. Duplicate the
+existing issues one, changing the ``issue`` class name to ``support``.
+6. Re-start your tracker and start using the new ``support`` class.
+Optionally, you might want to restrict the users able to access this new
+class to just the users with a new "SysAdmin" Role. To do this, we add
+some security declarations::
+db.security.addPermissionToRole('SysAdmin', 'View', 'support')
+db.security.addPermissionToRole('SysAdmin', 'Create', 'support')
+db.security.addPermissionToRole('SysAdmin', 'Edit', 'support')
+You would then (as an "admin" user) edit the details of the appropriate
+users, and add "SysAdmin" to their Roles list.
+Alternatively, you might want to change the Edit/View permissions granted
+for the ``issue`` class so that it's only available to users with the "System"
+or "Developer" Role, and then the new class you're adding is available to
+all with the "User" Role.
+.. _external-authentication:
+Using External User Databases
+-----------------------------
+Using an external password validation source
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+.. note:: You will need to either have an "admin" user in your external
+password source *or* have one of your regular users have
+the Admin Role assigned. If you need to assign the Role *after*
+making the changes below, you may use the ``roundup-admin``
+program to edit a user's details.
+We have a centrally-managed password changing system for our users. This
+results in a UN*X passwd-style file that we use for verification of
+users. Entries in the file consist of ``name:password`` where the
+password is encrypted using the standard UN*X ``crypt()`` function (see
+the ``crypt`` module in your Python distribution). An example entry
+would be::
+admin:aamrgyQfDFSHw
+Each user of Roundup must still have their information stored in the Roundup
+database - we just use the passwd file to check their password. To do this, we
+need to override the standard ``verifyPassword`` method defined in
+``roundup.cgi.actions.LoginAction`` and register the new class. The
+following is added as ``externalpassword.py`` in the tracker ``extensions``
+directory::
+import os, crypt
+from roundup.cgi.actions import LoginAction
+class ExternalPasswordLoginAction(LoginAction):
+def verifyPassword(self, userid, password):
+'''Look through the file, line by line, looking for a
+name that matches.
+'''
+# get the user's username
+username = self.db.user.get(userid, 'username')
+# the passwords are stored in the "passwd.txt" file in the
+# tracker home
+file = os.path.join(self.db.config.TRACKER_HOME, 'passwd.txt')
+# see if we can find a match
+for ent in [line.strip().split(':') for line in
+open(file).readlines()]:
+if ent[0] == username:
+return crypt.crypt(password, ent[1][:2]) == ent[1]
+# user doesn't exist in the file
+return 0
+def init(instance):
+instance.registerAction('login', ExternalPasswordLoginAction)
+You should also remove the redundant password fields from the ``user.item``
+template.
+Using a UN*X passwd file as the user database
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+On some systems the primary store of users is the UN*X passwd file. It
+holds information on users such as their username, real name, password
+and primary user group.
+Roundup can use this store as its primary source of user information,
+but it needs additional information too - email address(es), Roundup
+Roles, vacation flags, Roundup hyperdb item ids, etc. Also, "retired"
+users must still exist in the user database, unlike some passwd files in
+which the users are removed when they no longer have access to a system.
+To make use of the passwd file, we therefore synchronise between the two
+user stores. We also use the passwd file to validate the user logins, as
+described in the previous example, `using an external password
+validation source`_. We keep the user lists in sync using a fairly
+simple script that runs once a day, or several times an hour if more
+immediate access is needed. In short, it:
+1. parses the passwd file, finding usernames, passwords and real names,
+2. compares that list to the current Roundup user list:
+a. entries no longer in the passwd file are *retired*
+b. entries with mismatching real names are *updated*
+c. entries only exist in the passwd file are *created*
+3. send an email to administrators to let them know what's been done.
+The retiring and updating are simple operations, requiring only a call
+to ``retire()`` or ``set()``. The creation operation requires more
+information though - the user's email address and their Roundup Roles.
+We're going to assume that the user's email address is the same as their
+login name, so we just append the domain name to that. The Roles are
+determined using the passwd group identifier - mapping their UN*X group
+to an appropriate set of Roles.
+The script to perform all this, broken up into its main components, is
+as follows. Firstly, we import the necessary modules and open the
+tracker we're to work on::
+import sys, os, smtplib
+from roundup import instance, date
+# open the tracker
+tracker_home = sys.argv[1]
+tracker = instance.open(tracker_home)
+Next we read in the *passwd* file from the tracker home::
+# read in the users from the "passwd.txt" file
+file = os.path.join(tracker_home, 'passwd.txt')
+users = [x.strip().split(':') for x in open(file).readlines()]
+Handle special users (those to ignore in the file, and those who don't
+appear in the file)::
+# users to not keep ever, pre-load with the users I know aren't
+# "real" users
+ignore = ['ekmmon', 'bfast', 'csrmail']
+# users to keep - pre-load with the roundup-specific users
+keep = ['comment_pool', 'network_pool', 'admin', 'dev-team',
+'cs_pool', 'anonymous', 'system_pool', 'automated']
+Now we map the UN*X group numbers to the Roles that users should have::
+roles = {
+'501': 'User,Tech',  # tech
+'502': 'User',       # finance
+'503': 'User,CSR',   # customer service reps
+'504': 'User',       # sales
+'505': 'User',       # marketing
+}
+Now we do all the work. Note that the body of the script (where we have
+the tracker database open) is wrapped in a ``try`` / ``finally`` clause,
+so that we always close the database cleanly when we're finished. So, we
+now do all the work::
+# open the database
+db = tracker.open('admin')
+try:
+# store away messages to send to the tracker admins
+msg = []
+# loop over the users list read in from the passwd file
+for user,passw,uid,gid,real,home,shell in users:
+if user in ignore:
+# this user shouldn't appear in our tracker
+continue
+keep.append(user)
+try:
+# see if the user exists in the tracker
+uid = db.user.lookup(user)
+# yes, they do - now check the real name for correctness
+if real != db.user.get(uid, 'realname'):
+db.user.set(uid, realname=real)
+msg.append('FIX %s - %s'%(user, real))
+except KeyError:
+# nope, the user doesn't exist
+db.user.create(username=user, realname=real,
+address='%s@ekit-inc.com'%user, roles=roles[gid])
+msg.append('ADD %s - %s (%s)'%(user, real, roles[gid]))
+# now check that all the users in the tracker are also in our
+# "keep" list - retire those who aren't
+for uid in db.user.list():
+user = db.user.get(uid, 'username')
+if user not in keep:
+db.user.retire(uid)
+msg.append('RET %s'%user)
+# if we did work, then send email to the tracker admins
+if msg:
+# create the email
+msg = '''Subject: %s user database maintenance
+%s
+'''%(db.config.TRACKER_NAME, '\n'.join(msg))
+# send the email
+smtp = smtplib.SMTP(db.config.MAILHOST)
+addr = db.config.ADMIN_EMAIL
+smtp.sendmail(addr, addr, msg)
+# now we're done - commit the changes
+db.commit()
+finally:
+# always close the database cleanly
+db.close()
+And that's it!
+Using an LDAP database for user information
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+A script that reads users from an LDAP store using
+https://pypi.org/project/python-ldap/ and then compares the list to the users in the
+Roundup user database would be pretty easy to write. You'd then have it run
+once an hour / day (or on demand if you can work that into your LDAP store
+workflow). See the example `Using a UN*X passwd file as the user database`_
+for more information about doing this.
+To authenticate off the LDAP store (rather than using the passwords in the
+Roundup user database) you'd use the same python-ldap module inside an
+extension to the cgi interface. You'd do this by overriding the method called
+``verifyPassword`` on the ``LoginAction`` class in your tracker's
+``extensions`` directory (see `using an external password validation
+source`_). The method is implemented by default as::
+def verifyPassword(self, userid, password):
+''' Verify the password that the user has supplied
+'''
+stored = self.db.user.get(self.userid, 'password')
+if password == stored:
+return 1
+if not password and not stored:
+return 1
+return 0
+So you could reimplement this as something like::
+def verifyPassword(self, userid, password):
+''' Verify the password that the user has supplied
+'''
+# look up some unique LDAP information about the user
+username = self.db.user.get(self.userid, 'username')
+# now verify the password supplied against the LDAP store
+Changes to Tracker Behaviour
+----------------------------
+.. index:: single: auditors; how to register (example)
+single: reactors; how to register (example)
+Preventing SPAM
+~~~~~~~~~~~~~~~
+The following detector code may be installed in your tracker's
+``detectors`` directory. It will block any messages being created that
+have HTML attachments (a very common vector for spam and phishing)
+and any messages that have more than 2 HTTP URLs in them. Just copy
+the following into ``detectors/anti_spam.py`` in your tracker::
+from roundup.exceptions import Reject
+def reject_html(db, cl, nodeid, newvalues):
+if newvalues['type'] == 'text/html':
+raise Reject('not allowed')
+def reject_manylinks(db, cl, nodeid, newvalues):
+content = newvalues['content']
+if content.count('http://') > 2:
+raise Reject('not allowed')
+def init(db):
+db.file.audit('create', reject_html)
+db.msg.audit('create', reject_manylinks)
+You may also wish to block image attachments if your tracker does not
+need that ability::
+if newvalues['type'].startswith('image/'):
+raise Reject('not allowed')
+Stop "nosy" messages going to people on vacation
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+When users go on vacation and set up vacation email bouncing, you'll
+start to see a lot of messages come back through Roundup "Fred is on
+vacation". Not very useful, and relatively easy to stop.
+1. add a "vacation" flag to your users::
+user = Class(db, "user",
+username=String(),   password=Password(),
+address=String(),    realname=String(),
+phone=String(),      organisation=String(),
+alternate_addresses=String(),
+roles=String(), queries=Multilink("query"),
+vacation=Boolean())
+2. So that users may edit the vacation flags, add something like the
+following to your ``user.item`` template::
+<tr>
+<th>On Vacation</th>
+<td tal:content="structure context/vacation/field">vacation</td>
+</tr>
+3. edit your detector ``nosyreactor.py`` so that the ``nosyreaction()``
+consists of::
+def nosyreaction(db, cl, nodeid, oldvalues):
+users = db.user
+messages = db.msg
+# send a copy of all new messages to the nosy list
+for msgid in determineNewMessages(cl, nodeid, oldvalues):
+try:
+# figure the recipient ids
+sendto = []
+seen_message = {}
+recipients = messages.get(msgid, 'recipients')
+for recipid in messages.get(msgid, 'recipients'):
+seen_message[recipid] = 1
+# figure the author's id, and indicate they've received
+# the message
+authid = messages.get(msgid, 'author')
+# possibly send the message to the author, as long as
+# they aren't anonymous
+if (db.config.MESSAGES_TO_AUTHOR == 'yes' and
+users.get(authid, 'username') != 'anonymous'):
+sendto.append(authid)
+seen_message[authid] = 1
+# now figure the nosy people who weren't recipients
+nosy = cl.get(nodeid, 'nosy')
+for nosyid in nosy:
+# Don't send nosy mail to the anonymous user (that
+# user shouldn't appear in the nosy list, but just
+# in case they do...)
+if users.get(nosyid, 'username') == 'anonymous':
+continue
+# make sure they haven't seen the message already
+if nosyid not in seen_message:
+# send it to them
+sendto.append(nosyid)
+recipients.append(nosyid)
+# generate a change note
+if oldvalues:
+note = cl.generateChangeNote(nodeid, oldvalues)
+else:
+note = cl.generateCreateNote(nodeid)
+# we have new recipients
+if sendto:
+# filter out the people on vacation
+sendto = [i for i in sendto
+if not users.get(i, 'vacation', 0)]
+# map userids to addresses
+sendto = [users.get(i, 'address') for i in sendto]
+# update the message's recipients list
+messages.set(msgid, recipients=recipients)
+# send the message
+cl.send_message(nodeid, msgid, note, sendto)
+except roundupdb.MessageSendError as message:
+raise roundupdb.DetectorError(message)
+Note that this is the standard nosy reaction code, with the small
+addition of::
+# filter out the people on vacation
+sendto = [i for i in sendto if not users.get(i, 'vacation', 0)]
+which filters out the users that have the vacation flag set to true.
+Adding in state transition control
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Sometimes tracker admins want to control the states to which users may
+move issues. You can do this by following these steps:
+1. make "status" a required variable. This is achieved by adding the
+following to the top of the form in the ``issue.item.html``
+template::
+<input type="hidden" name="@required" value="status">
+This will force users to select a status.
+2. add a Multilink property to the status class::
+stat = Class(db, "status", ... , transitions=Multilink('status'),
+...)
+and then edit the statuses already created, either:
+a. through the web using the class list -> status class editor, or
+b. using the ``roundup-admin`` "set" command.
+3. add an auditor module ``checktransition.py`` in your tracker's
+``detectors`` directory, for example::
+def checktransition(db, cl, nodeid, newvalues):
+''' Check that the desired transition is valid for the "status"
+property.
+'''
+if 'status' not in newvalues:
+return
+current = cl.get(nodeid, 'status')
+new = newvalues['status']
+if new == current:
+return
+ok = db.status.get(current, 'transitions')
+if new not in ok:
+raise ValueError('Status not allowed to move from "%s" to "%s"'%(
+db.status.get(current, 'name'), db.status.get(new, 'name')))
+def init(db):
+db.issue.audit('set', checktransition)
+4. in the ``issue.item.html`` template, change the status editing bit
+from::
+<th>Status</th>
+<td tal:content="structure context/status/menu">status</td>
+to::
+<th>Status</th>
+<td>
+<select tal:condition="context/id" name="status">
+<tal:block tal:define="ok context/status/transitions"
+tal:repeat="state db/status/list">
+<option tal:condition="python:state.id in ok"
+tal:attributes="
+value state/id;
+selected python:state.id == context.status.id"
+tal:content="state/name"></option>
+</tal:block>
+</select>
+<tal:block tal:condition="not:context/id"
+tal:replace="structure context/status/menu" />
+</td>
+which displays only the allowed status to transition to.
+Blocking issues that depend on other issues
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+We needed the ability to mark certain issues as "blockers" - that is,
+they can't be resolved until another issue (the blocker) they rely on is
+resolved. To achieve this:
+1. Create a new property on the ``issue`` class:
+``blockers=Multilink("issue")``. To do this, edit the definition of
+this class in your tracker's ``schema.py`` file. Change this::
+issue = IssueClass(db, "issue",
+assignedto=Link("user"), keyword=Multilink("keyword"),
+priority=Link("priority"), status=Link("status"))
+to this, adding the blockers entry::
+issue = IssueClass(db, "issue",
+blockers=Multilink("issue"),
+assignedto=Link("user"), keyword=Multilink("keyword"),
+priority=Link("priority"), status=Link("status"))
+2. Add the new ``blockers`` property to the ``issue.item.html`` edit
+page, using something like::
+<th>Waiting On</th>
+<td>
+<span tal:replace="structure python:context.blockers.field(showid=1,
+size=20)" />
+<span tal:replace="structure python:db.issue.classhelp('id,title',
+property='blockers')" />
+<span tal:condition="context/blockers"
+tal:repeat="blk context/blockers">
+<br>View: <a tal:attributes="href string:issue${blk/id}"
+tal:content="blk/id"></a>
+</span>
+</td>
+You'll need to fiddle with your item page layout to find an
+appropriate place to put it - I'll leave that fun part up to you.
+Just make sure it appears in the first table, possibly somewhere near
+the "superseders" field.
+3. Create a new detector module (see below) which enforces the rules:
+- issues may not be resolved if they have blockers
+- when a blocker is resolved, it's removed from issues it blocks
+The contents of the detector should be something like this::
+def blockresolution(db, cl, nodeid, newvalues):
+''' If the issue has blockers, don't allow it to be resolved.
+'''
+if nodeid is None:
+blockers = []
+else:
+blockers = cl.get(nodeid, 'blockers')
+blockers = newvalues.get('blockers', blockers)
+# don't do anything if there's no blockers or the status hasn't
+# changed
+if not blockers or 'status' not in newvalues:
+return
+# get the resolved state ID
+resolved_id = db.status.lookup('resolved')
+# format the info
+u = db.config.TRACKER_WEB
+s = ', '.join(['<a href="%sissue%s">%s</a>'%(
+u,id,id) for id in blockers])
+if len(blockers) == 1:
+s = 'issue %s is'%s
+else:
+s = 'issues %s are'%s
+# ok, see if we're trying to resolve
+if newvalues['status'] == resolved_id:
+raise ValueError("This issue can't be resolved until %s resolved."%s)
+def resolveblockers(db, cl, nodeid, oldvalues):
+''' When we resolve an issue that's a blocker, remove it from the
+blockers list of the issue(s) it blocks.
+'''
+newstatus = cl.get(nodeid,'status')
+# no change?
+if oldvalues.get('status', None) == newstatus:
+return
+resolved_id = db.status.lookup('resolved')
+# interesting?
+if newstatus != resolved_id:
+return
+# yes - find all the blocked issues, if any, and remove me from
+# their blockers list
+issues = cl.find(blockers=nodeid)
+for issueid in issues:
+blockers = cl.get(issueid, 'blockers')
+if nodeid in blockers:
+blockers.remove(nodeid)
+cl.set(issueid, blockers=blockers)
+def init(db):
+# might, in an obscure situation, happen in a create
+db.issue.audit('create', blockresolution)
+db.issue.audit('set', blockresolution)
+# can only happen on a set
+db.issue.react('set', resolveblockers)
+Put the above code in a file called "blockers.py" in your tracker's
+"detectors" directory.
+4. Finally, and this is an optional step, modify the tracker web page
+URLs so they filter out issues with any blockers. You do this by
+adding an additional filter on "blockers" for the value "-1". For
+example, the existing "Show All" link in the "page" template (in the
+tracker's "html" directory) looks like this::
+<a href="#"
+tal:attributes="href python:request.indexargs_url('issue', {
+'@sort': '-activity',
+'@group': 'priority',
+'@filter': 'status',
+'@columns': columns_showall,
+'@search_text': '',
+'status': status_notresolved,
+'@dispname': i18n.gettext('Show All'),
+})"
+i18n:translate="">Show All</a><br>
+modify it to add the "blockers" info to the URL (note, both the
+"@filter" *and* "blockers" values must be specified)::
+<a href="#"
+tal:attributes="href python:request.indexargs_url('issue', {
+'@sort': '-activity',
+'@group': 'priority',
+'@filter': 'status,blockers',
+'@columns': columns_showall,
+'@search_text': '',
+'status': status_notresolved,
+'blockers': '-1',
+'@dispname': i18n.gettext('Show All'),
+})"
+i18n:translate="">Show All</a><br>
+The above examples are line-wrapped on the trailing & and should
+be unwrapped.
+That's it. You should now be able to set blockers on your issues. Note
+that if you want to know whether an issue has any other issues dependent
+on it (i.e. it's in their blockers list) you can look at the journal
+history at the bottom of the issue page - look for a "link" event to
+another issue's "blockers" property.
+Add users to the nosy list based on the keyword
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Let's say we need the ability to automatically add users to the nosy
+list based
+on the occurance of a keyword. Every user should be allowed to edit their
+own list of keywords for which they want to be added to the nosy list.
+Below, we'll show that this change can be done with minimal
+understanding of the Roundup system, using only copy and paste.
+This requires three changes to the tracker: a change in the database to
+allow per-user recording of the lists of keywords for which he wants to
+be put on the nosy list, a change in the user view allowing them to edit
+this list of keywords, and addition of an auditor which updates the nosy
+list when a keyword is set.
+Adding the nosy keyword list
+::::::::::::::::::::::::::::
+The change to make in the database, is that for any user there should be a list
+of keywords for which he wants to be put on the nosy list. Adding a
+``Multilink`` of ``keyword`` seems to fullfill this. As such, all that has to
+be done is to add a new field to the definition of ``user`` within the file
+``schema.py``.  We will call this new field ``nosy_keywords``, and the updated
+definition of user will be::
+user = Class(db, "user",
+username=String(),   password=Password(),
+address=String(),    realname=String(),
+phone=String(),      organisation=String(),
+alternate_addresses=String(),
+queries=Multilink('query'), roles=String(),
+timezone=String(),
+nosy_keywords=Multilink('keyword'))
+Changing the user view to allow changing the nosy keyword list
+::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
+We want any user to be able to change the list of keywords for which
+he will by default be added to the nosy list. We choose to add this
+to the user view, as is generated by the file ``html/user.item.html``.
+We can easily
+see that the keyword field in the issue view has very similar editing
+requirements as our nosy keywords, both being lists of keywords. As
+such, we look for Keywords in ``issue.item.html``, and extract the
+associated parts from there. We add this to ``user.item.html`` at the
+bottom of the list of viewed items (i.e. just below the 'Alternate
+E-mail addresses' in the classic template)::
+<tr>
+<th>Nosy Keywords</th>
+<td>
+<span tal:replace="structure context/nosy_keywords/field" />
+<span tal:replace="structure python:db.keyword.classhelp(property='nosy_keywords')" />
+</td>
+</tr>
+Addition of an auditor to update the nosy list
+::::::::::::::::::::::::::::::::::::::::::::::
+The more difficult part is the logic to add
+the users to the nosy list when required.
+We choose to perform this action whenever the keywords on an
+item are set (this includes the creation of items).
+Here we choose to start out with a copy of the
+``detectors/nosyreaction.py`` detector, which we copy to the file
+``detectors/nosy_keyword_reaction.py``.
+This looks like a good start as it also adds users
+to the nosy list. A look through the code reveals that the
+``nosyreaction`` function actually sends the e-mail.
+We don't need this. Therefore, we can change the ``init`` function to::
+def init(db):
+db.issue.audit('create', update_kw_nosy)
+db.issue.audit('set', update_kw_nosy)
+After that, we rename the ``updatenosy`` function to ``update_kw_nosy``.
+The first two blocks of code in that function relate to setting
+``current`` to a combination of the old and new nosy lists. This
+functionality is left in the new auditor. The following block of
+code, which handled adding the assignedto user(s) to the nosy list in
+``updatenosy``, should be replaced by a block of code to add the
+interested users to the nosy list. We choose here to loop over all
+new keywords, than looping over all users,
+and assign the user to the nosy list when the keyword occurs in the user's
+``nosy_keywords``. The next part in ``updatenosy`` -- adding the author
+and/or recipients of a message to the nosy list -- is obviously not
+relevant here and is thus deleted from the new auditor. The last
+part, copying the new nosy list to ``newvalues``, can stay as is.
+This results in the following function::
+def update_kw_nosy(db, cl, nodeid, newvalues):
+'''Update the nosy list for changes to the keywords
+'''
+# nodeid will be None if this is a new node
+current = {}
+if nodeid is None:
+ok = ('new', 'yes')
+else:
+ok = ('yes',)
+# old node, get the current values from the node if they haven't
+# changed
+if 'nosy' not in newvalues:
+nosy = cl.get(nodeid, 'nosy')
+for value in nosy:
+if value not in current:
+current[value] = 1
+# if the nosy list changed in this transaction, init from the new value
+if 'nosy' in newvalues:
+nosy = newvalues.get('nosy', [])
+for value in nosy:
+if not db.hasnode('user', value):
+continue
+if value not in current:
+current[value] = 1
+# add users with keyword in nosy_keywords to the nosy list
+if 'keyword' in newvalues and newvalues['keyword'] is not None:
+keyword_ids = newvalues['keyword']
+for keyword in keyword_ids:
+# loop over all users,
+# and assign user to nosy when keyword in nosy_keywords
+for user_id in db.user.list():
+nosy_kw = db.user.get(user_id, "nosy_keywords")
+found = 0
+for kw in nosy_kw:
+if kw == keyword:
+found = 1
+if found:
+current[user_id] = 1
+# that's it, save off the new nosy list
+newvalues['nosy'] = list(current.keys())
+These two function are the only ones needed in the file.
+TODO: update this example to use the ``find()`` Class method.
+Caveats
+:::::::
+A few problems with the design here can be noted:
+Multiple additions
+When a user, after automatic selection, is manually removed
+from the nosy list, he is added to the nosy list again when the
+keyword list of the issue is updated. A better design might be
+to only check which keywords are new compared to the old list
+of keywords, and only add users when they have indicated
+interest on a new keyword.
+The code could also be changed to only trigger on the ``create()``
+event, rather than also on the ``set()`` event, thus only setting
+the nosy list when the issue is created.
+Scalability
+In the auditor, there is a loop over all users. For a site with
+only few users this will pose no serious problem; however, with
+many users this will be a serious performance bottleneck.
+A way out would be to link from the keywords to the users who
+selected these keywords as nosy keywords. This will eliminate the
+loop over all users. See the ``rev_multilink`` attribute to make
+this easier.
+Restricting updates that arrive by email
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Roundup supports multiple update methods:
+1. command line
+2. plain email
+3. pgp signed email
+4. web access
+in some cases you may need to prevent changes to properties by some of
+these methods. For example you can set up issues that are viewable
+only by people on the nosy list. So you must prevent unauthenticated
+changes to the nosy list.
+Since plain email can be easily forged, it does not provide sufficient
+authentication in this senario.
+To prevent this we can add a detector that audits the source of the
+transaction and rejects the update if it changes the nosy list.
+Create the detector (auditor) module and add it to the detectors
+directory of your tracker::
+from roundup import roundupdb, hyperdb
+from roundup.mailgw import Unauthorized
+def restrict_nosy_changes(db, cl, nodeid, newvalues):
+'''Do not permit changes to nosy via email.'''
+if 'nosy' not in newvalues:
+# the nosy field has not changed so no need to check.
+return
+if db.tx_Source in ['web', 'rest', 'xmlrpc', 'email-sig-openpgp', 'cli' ]:
+	   # if the source of the transaction is from an authenticated
+	   # source or a privileged process allow the transaction.
+	   # Other possible sources: 'email'
+	   return
+# otherwise raise an error
+raise Unauthorized( \
+	   'Changes to nosy property not allowed via %s for this issue.'%\
+tx_Source)
+def init(db):
+''' Install restrict_nosy_changes to run after other auditors.
+Allow initial creation email to set nosy.
+So don't execute: db.issue.audit('create', requestedbyauditor)
+Set priority to 110 to run this auditor after other auditors
+that can cause nosy to change.
+'''
+db.issue.audit('set', restrict_nosy_changes, 110)
+This detector (auditor) will prevent updates to the nosy field if it
+arrives by email. Since it runs after other auditors (due to the
+priority of 110), it will also prevent changes to the nosy field that
+are done by other auditors if triggered by an email.
+Note that db.tx_Source was not present in roundup versions before
+1.4.22, so you must be running a newer version to use this detector.
+Read the CHANGES.txt document in the roundup source code for further
+details on tx_Source.
+Changes to Security and Permissions
+-----------------------------------
+Restricting the list of users that are assignable to a task
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+1. In your tracker's ``schema.py``, create a new Role, say "Developer"::
+db.security.addRole(name='Developer', description='A developer')
+2. Just after that, create a new Permission, say "Fixer", specific to
+"issue"::
+p = db.security.addPermission(name='Fixer', klass='issue',
+description='User is allowed to be assigned to fix issues')
+3. Then assign the new Permission to your "Developer" Role::
+db.security.addPermissionToRole('Developer', p)
+4. In the issue item edit page (``html/issue.item.html`` in your tracker
+directory), use the new Permission in restricting the "assignedto"
+list::
+<select name="assignedto">
+<option value="-1">- no selection -</option>
+<tal:block tal:repeat="user db/user/list">
+<option tal:condition="python:user.hasPermission(
+'Fixer', context._classname)"
+tal:attributes="
+value user/id;
+selected python:user.id == context.assignedto"
+tal:content="user/realname"></option>
+</tal:block>
+</select>
+For extra security, you may wish to setup an auditor to enforce the
+Permission requirement (install this as ``assignedtoFixer.py`` in your
+tracker ``detectors`` directory)::
+def assignedtoMustBeFixer(db, cl, nodeid, newvalues):
+''' Ensure the assignedto value in newvalues is used with the
+Fixer Permission
+'''
+if 'assignedto' not in newvalues:
+# don't care
+return
+# get the userid
+userid = newvalues['assignedto']
+if not db.security.hasPermission('Fixer', userid, cl.classname):
+raise ValueError('You do not have permission to edit %s'%cl.classname)
+def init(db):
+db.issue.audit('set', assignedtoMustBeFixer)
+db.issue.audit('create', assignedtoMustBeFixer)
+So now, if an edit action attempts to set "assignedto" to a user that
+doesn't have the "Fixer" Permission, the error will be raised.
+Users may only edit their issues
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+In this case, users registering themselves are granted Provisional
+access, meaning they
+have access to edit the issues they submit, but not others. We create a new
+Role called "Provisional User" which is granted to newly-registered users,
+and has limited access. One of the Permissions they have is the new "Edit
+Own" on issues (regular users have "Edit".)
+First up, we create the new Role and Permission structure in
+``schema.py``::
+#
+# New users not approved by the admin
+#
+db.security.addRole(name='Provisional User',
+description='New user registered via web or email')
+# These users need to be able to view and create issues but only edit
+# and view their own
+db.security.addPermissionToRole('Provisional User', 'Create', 'issue')
+def own_issue(db, userid, itemid):
+'''Determine whether the userid matches the creator of the issue.'''
+return userid == db.issue.get(itemid, 'creator')
+p = db.security.addPermission(name='Edit', klass='issue',
+check=own_issue, description='Can only edit own issues')
+db.security.addPermissionToRole('Provisional User', p)
+p = db.security.addPermission(name='View', klass='issue',
+check=own_issue, description='Can only view own issues')
+db.security.addPermissionToRole('Provisional User', p)
+# This allows the interface to get the names of the properties
+# in the issue. Used for selecting sorting and grouping
+# on the index page.
+p = db.security.addPermission(name='Search', klass='issue')
+db.security.addPermissionToRole ('Provisional User', p)
+# Assign the Permissions for issue-related classes
+for cl in 'file', 'msg', 'query', 'keyword':
+db.security.addPermissionToRole('Provisional User', 'View', cl)
+db.security.addPermissionToRole('Provisional User', 'Edit', cl)
+db.security.addPermissionToRole('Provisional User', 'Create', cl)
+for cl in 'priority', 'status':
+db.security.addPermissionToRole('Provisional User', 'View', cl)
+# and give the new users access to the web and email interface
+db.security.addPermissionToRole('Provisional User', 'Web Access')
+db.security.addPermissionToRole('Provisional User', 'Email Access')
+# make sure they can view & edit their own user record
+def own_record(db, userid, itemid):
+'''Determine whether the userid matches the item being accessed.'''
+return userid == itemid
+p = db.security.addPermission(name='View', klass='user', check=own_record,
+description="User is allowed to view their own user details")
+db.security.addPermissionToRole('Provisional User', p)
+p = db.security.addPermission(name='Edit', klass='user', check=own_record,
+description="User is allowed to edit their own user details")
+db.security.addPermissionToRole('Provisional User', p)
+Then, in ``config.ini``, we change the Role assigned to newly-registered
+users, replacing the existing ``'User'`` values::
+[main]
+...
+new_web_user_roles = Provisional User
+new_email_user_roles = Provisional User
+All users may only view and edit issues, files and messages they create
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Replace the standard "classic" tracker View and Edit Permission assignments
+for the "issue", "file" and "msg" classes with the following::
+def checker(klass):
+def check(db, userid, itemid, klass=klass):
+return db.getclass(klass).get(itemid, 'creator') == userid
+return check
+for cl in 'issue', 'file', 'msg':
+p = db.security.addPermission(name='View', klass=cl,
+check=checker(cl),
+description='User can view only if creator.')
+db.security.addPermissionToRole('User', p)
+p = db.security.addPermission(name='Edit', klass=cl,
+check=checker(cl),
+description='User can edit only if creator.')
+db.security.addPermissionToRole('User', p)
+db.security.addPermissionToRole('User', 'Create', cl)
+# This allows the interface to get the names of the properties
+# in the issue. Used for selecting sorting and grouping
+# on the index page.
+p = db.security.addPermission(name='Search', klass='issue')
+db.security.addPermissionToRole ('User', p)
+Moderating user registration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+You could set up new-user moderation in a public tracker by:
+1. creating a new highly-restricted user role "Pending",
+2. set the config new_web_user_roles and/or new_email_user_roles to that
+role,
+3. have an auditor that emails you when new users are created with that
+role using roundup.mailer
+4. edit the role to "User" for valid users.
+Some simple javascript might help in the last step. If you have high volume
+you could search for all currently-Pending users and do a bulk edit of all
+their roles at once (again probably with some simple javascript help).
+Changes to the Web User Interface
 ---------------------------------
-Some examples are in the :ref:`CustomExamples` section below.
+Adding action links to the index page
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Also the `Roundup wiki`_ has additional examples of how schemas can be
-customised to add new functionality.
+Add a column to the ``item.index.html`` template.
-.. _Roundup wiki:
+Resolving the issue::
-https://wiki.roundup-tracker.org/
+<a tal:attributes="href
-.. index:: !detectors
+string:issue${i/id}?:status=resolved&:action=edit">resolve</a>
-Detectors - adding behaviour to your tracker
+"Take" the issue::
-============================================
-.. _detectors:
+<a tal:attributes="href
+string:issue${i/id}?:assignedto=${request/user/id}&:action=edit">take</a>
-Detectors are initialised every time you open your tracker database, so
-you're free to add and remove them any time, even after the database is
+... and so on.
-initialised via the ``roundup-admin initialise`` command.
+Colouring the rows in the issue index according to priority
-The detectors in your tracker fire *before* (**auditors**) and *after*
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-(**reactors**) changes to the contents of your database. They are Python
-modules that sit in your tracker's ``detectors`` directory. You will
+A simple ``tal:attributes`` statement will do the bulk of the work here. In
-have some installed by default - have a look. You can write new
+the ``issue.index.html`` template, add this to the ``<tr>`` that
-detectors or modify the existing ones. The existing detectors installed
+displays the rows of data::
-for you are:
+<tr tal:attributes="class string:priority-${i/priority/plain}">
-.. index:: detectors; installed
+and then in your stylesheet (``style.css``) specify the colouring for the
-**nosyreaction.py**
+different priorities, as follows::
-This provides the automatic nosy list maintenance and email sending.
-The nosy reactor (``nosyreaction``) fires when new messages are added
+tr.priority-critical td {
-to issues. The nosy auditor (``updatenosy``) fires when issues are
+background-color: red;
-changed, and figures out what changes need to be made to the nosy list
+}
-(such as adding new authors, etc.)
-**statusauditor.py**
+tr.priority-urgent td {
-This provides the ``chatty`` auditor which changes the issue status
+background-color: orange;
-from ``unread`` or ``closed`` to ``chatting`` if new messages appear.
+}
-It also provides the ``presetunread`` auditor which pre-sets the
-status to ``unread`` on new items if the status isn't explicitly
+and so on, with far less offensive colours :)
-defined.
-**messagesummary.py**
+Editing multiple items in an index view
-Generates the ``summary`` property for new messages based on the message
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-content.
-**userauditor.py**
+To edit the status of all items in the item index view, edit the
-Verifies the content of some of the user fields (email addresses and
+``issue.index.html``:
-roles lists).
+1. add a form around the listing table (separate from the existing
-If you don't want this default behaviour, you're completely free to change
+index-page form), so at the top it reads::
-or remove these detectors.
+<form method="POST" tal:attributes="action request/classname">
-See the detectors section in the `design document`__ for details of the
+<table class="list">
-interface for detectors.
+and at the bottom of that table::
-__ design.html
+</table>
+</form
-.. index:: detectors; writing api
+making sure you match the ``</table>`` from the list table, not the
-Detector API
+navigation table or the subsequent form table.
-------------
+2. in the display for the issue property, change::
-.. index:: pair: detectors; auditors
-single: auditors; function signature
+<td tal:condition="request/show/status"
-single: auditors; defining
+tal:content="python:i.status.plain() or default">&nbsp;</td>
-single: auditors; arguments
+to::
-Auditors are called with the arguments::
+<td tal:condition="request/show/status"
-audit(db, cl, itemid, newdata)
+tal:content="structure i/status/field">&nbsp;</td>
-where ``db`` is the database, ``cl`` is an instance of Class or
+this will result in an edit field for the status property.
-IssueClass within the database, and ``newdata`` is a dictionary mapping
-property names to values.
+3. after the ``tal:block`` which lists the index items (marked by
+``tal:repeat="i batch"``) add a new table row::
-For a ``create()`` operation, the ``itemid`` argument is None and
-newdata contains all of the initial property values with which the item
+<tr>
-is about to be created.
+<td tal:attributes="colspan python:len(request.columns)">
+<input name="@csrf" type="hidden"
-For a ``set()`` operation, newdata contains only the names and values of
+tal:attributes="value python:utils.anti_csrf_nonce()">
-properties that are about to be changed.
+<input type="submit" value=" Save Changes ">
+<input type="hidden" name="@action" value="edit">
-For a ``retire()`` or ``restore()`` operation, newdata is None.
+<tal:block replace="structure request/indexargs_form" />
+</td>
-.. index:: pair: detectors; reactor
+</tr>
-single: reactors; function signature
-single: reactors; defining
+which gives us a submit button, indicates that we are performing an
-single: reactors; arguments
+edit on any changed statuses, and provides a defense against cross
+site request forgery attacks.
-Reactors are called with the arguments::
+The final ``tal:block`` will make sure that the current index view
-react(db, cl, itemid, olddata)
+parameters (filtering, columns, etc) will be used in rendering the
+next page (the results of the editing).
-where ``db`` is the database, ``cl`` is an instance of Class or
-IssueClass within the database, and ``olddata`` is a dictionary mapping
-property names to values.
+Displaying only message summaries in the issue display
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-For a ``create()`` operation, the ``itemid`` argument is the id of the
-newly-created item and ``olddata`` is None.
+Alter the ``issue.item`` template section for messages to::
-For a ``set()`` operation, ``olddata`` contains the names and previous
+<table class="messages" tal:condition="context/messages">
-values of properties that were changed.
+<tr><th colspan="5" class="header">Messages</th></tr>
+<tr tal:repeat="msg context/messages">
-For a ``retire()`` or ``restore()`` operation, ``itemid`` is the id of
+<td><a tal:attributes="href string:msg${msg/id}"
-the retired or restored item and ``olddata`` is None.
+tal:content="string:msg${msg/id}"></a></td>
+<td tal:content="msg/author">author</td>
-.. index:: detectors; additional
+<td class="date" tal:content="msg/date/pretty">date</td>
+<td tal:content="msg/summary">summary</td>
-Additional Detectors Ready For Use
+<td>
-----------------------------------
+<a tal:attributes="href string:?@remove@messages=${msg/id}&@action=edit">
+remove</a>
-Sample additional detectors that have been found useful will appear in
+</td>
-the ``'detectors'`` directory of the Roundup distribution. If you want
+</tr>
-to use one, copy it to the ``'detectors'`` of your tracker instance:
+</table>
-**irker.py**
-This detector sends notification on IRC through an irker daemon
+Enabling display of either message summaries or the entire messages
-(http://www.catb.org/esr/irker/) when issues are created or messages
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-are added.  In order to use it you need to install irker, start the
-irkerd daemon, and add an ``[irker]`` section in ``detectors/config.ini``
+This is pretty simple - all we need to do is copy the code from the
-that contains a comma-separated list of channels where the messages should
+example `displaying only message summaries in the issue display`_ into
-be sent, e.g. ``channels = irc://chat.freenode.net/channelname``.
+our template alongside the summary display, and then introduce a switch
-**newissuecopy.py**
+that shows either the one or the other. We'll use a new form variable,
-This detector sends an email to a team address whenever a new issue is
+``@whole_messages`` to achieve this::
-created. The address is hard-coded into the detector, so edit it
-before you use it (look for the text 'team@team.host') or you'll get
+<table class="messages" tal:condition="context/messages">
-email errors!
+<tal:block tal:condition="not:request/form/@whole_messages/value | python:0">
-**creator_resolution.py**
+<tr><th colspan="3" class="header">Messages</th>
-Catch attempts to set the status to "resolved" - if the assignedto
+<th colspan="2" class="header">
-user isn't the creator, then set the status to "confirm-done". Note that
+<a href="?@whole_messages=yes">show entire messages</a>
-"classic" Roundup doesn't have that status, so you'll have to add it. If
+</th>
-you don't want to though, it'll just use "in-progress" instead.
+</tr>
-**email_auditor.py**
+<tr tal:repeat="msg context/messages">
-If a file added to an issue is of type message/rfc822, we tack on the
+<td><a tal:attributes="href string:msg${msg/id}"
-extension .eml.
+tal:content="string:msg${msg/id}"></a></td>
-The reason for this is that Microsoft Internet Explorer will not open
+<td tal:content="msg/author">author</td>
-things with a .eml attachment, as they deem it 'unsafe'. Worse yet,
+<td class="date" tal:content="msg/date/pretty">date</td>
-they'll just give you an incomprehensible error message. For more
+<td tal:content="msg/summary">summary</td>
-information, see the detector code - it has a length explanation.
+<td>
+<a tal:attributes="href string:?@remove@messages=${msg/id}&@action=edit">remove</a>
+</td>
-.. index:: auditors; rules for use
+</tr>
-single: reactors; rules for use
+</tal:block>
-Auditor or Reactor?
+<tal:block tal:condition="request/form/@whole_messages/value | python:0">
--------------------
+<tr><th colspan="2" class="header">Messages</th>
+<th class="header">
-Generally speaking, the following rules should be observed:
+<a href="?@whole_messages=">show only summaries</a>
+</th>
-**Auditors**
+</tr>
-Are used for `vetoing creation of or changes to items`_. They might
+<tal:block tal:repeat="msg context/messages">
-also make automatic changes to item properties.
+<tr>
-**Reactors**
+<th tal:content="msg/author">author</th>
-Detect changes in the database and react accordingly. They should avoid
+<th class="date" tal:content="msg/date/pretty">date</th>
-making changes to the database where possible, as this could create
+<th style="text-align: right">
-detector loops.
+(<a tal:attributes="href string:?@remove@messages=${msg/id}&@action=edit">remove</a>)
+</th>
+</tr>
-Vetoing creation of or changes to items
+<tr><td colspan="3" tal:content="msg/content"></td></tr>
----------------------------------------
+</tal:block>
+</tal:block>
-Auditors may raise the ``Reject`` exception to prevent the creation of
+</table>
-or changes to items in the database.  The mail gateway, for example, will
-not attach files or messages to issues when the creation of those files or
-messages are prevented through the ``Reject`` exception. It'll also not create
+Setting up a "wizard" (or "druid") for controlled adding of issues
-users if that creation is ``Reject``'ed too.
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-To use, simply add at the top of your auditor::
+1. Set up the page templates you wish to use for data input. My wizard
+is going to be a two-step process: first figuring out what category
-from roundup.exceptions import Reject
+of issue the user is submitting, and then getting details specific to
+that category. The first page includes a table of help, explaining
-And then when your rejection criteria have been detected, simply::
+what the category names mean, and then the core of the form::
-raise Reject('Description of error')
+<form method="POST" onSubmit="return submit_once()"
+enctype="multipart/form-data">
-Error messages raised with ``Reject`` automatically have any HTML content
+<input name="@csrf" type="hidden"
-escaped before being displayed to the user. To display an error message to the
+tal:attributes="value python:utils.anti_csrf_nonce()">
-user without performing any HTML escaping the ``RejectRaw`` should be used. All
+<input type="hidden" name="@template" value="add_page1">
-security implications should be carefully considering before using
+<input type="hidden" name="@action" value="page1_submit">
-``RejectRaw``.
+<strong>Category:</strong>
+<tal:block tal:replace="structure context/category/menu" />
-Generating email from Roundup
+<input type="submit" value="Continue">
------------------------------
+</form>
-The module ``roundup.mailer`` contains most of the nuts-n-bolts required
+The next page has the usual issue entry information, with the
-to generate email messages from Roundup.
+addition of the following form fragments::
-In addition, the ``IssueClass`` methods ``nosymessage()`` and
+<form method="POST" onSubmit="return submit_once()"
-``send_message()`` are used to generate nosy messages, and may generate
+enctype="multipart/form-data"
-messages which only consist of a change note (ie. the message id parameter
+tal:condition="context/is_edit_ok"
-is not required - this is referred to as a "System Message" because it
+tal:define="cat request/form/category/value">
-comes from "the system" and not a user).
+<input name="@csrf" type="hidden"
+tal:attributes="value python:utils.anti_csrf_nonce()">
-.. index:: extensions
+<input type="hidden" name="@template" value="add_page2">
-.. index:: extensions; add python functions to tracker
+<input type="hidden" name="@required" value="title">
+<input type="hidden" name="category" tal:attributes="value cat">
-Extensions - adding capabilities to your tracker
+.
-================================================
+.
-.. _extensions:
+.
+</form>
-While detectors_ add new behavior by reacting to changes in tracked
-objects, `extensions` add new actions and utilities to Roundup, which
+Note that later in the form, I use the value of "cat" to decide which
-are mostly used to enhance web interface.
+form elements should be displayed. For example::
-You can create an extension by creating Python file in your tracker
+<tal:block tal:condition="python:cat in '6 10 13 14 15 16 17'.split()">
-``extensions`` directory.  All files from this dir are loaded when
+<tr>
-tracker instance is created, at which point it calls ``init(instance)``
+<th>Operating System</th>
-from each file supplying itself as a first argument.
+<td tal:content="structure context/os/field"></td>
+</tr>
-Note that at this point web interface is not loaded, but extensions still
+<tr>
-can register actions for in tracker instance. This may be fixed in
+<th>Web Browser</th>
-Roundup 1.6 by introducing ``init_web(client)`` callback or a more
+<td tal:content="structure context/browser/field"></td>
-flexible extension point mechanism.
+</tr>
+</tal:block>
-* ``instance.registerUtil`` is used for adding `templating utilities`_
+... the above section will only be displayed if the category is one
-(see `adding a time log to your issues`_ for an example)
+of 6, 10, 13, 14, 15, 16 or 17.
-* ``instance.registerAction`` is used to add more actions to instance
+3. Determine what actions need to be taken between the pages - these are
-and to web interface. See `Defining new web actions`_ for details.
+usually to validate user choices and determine what page is next. Now encode
-Generic action can be added by inheriting from ``action.Action``
+those actions in a new ``Action`` class (see
-instead of ``cgi.action.Action``.
+`defining new web actions <reference.html#defining-new-web-actions>`_)::
-interfaces.py - hooking into the core of roundup
+from roundup.cgi.actions import Action
-================================================
-.. _interfaces.py:
+class Page1SubmitAction(Action):
+def handle(self):
-There is a magic trick for hooking into the core of roundup. Using
+''' Verify that the user has selected a category, and then move
-this you can:
+on to page 2.
+'''
-* modify class data structures
+category = self.form['category'].value
-* monkey patch core code to add new functionality
+if category == '-1':
-* modify the email gateway
+self.client.add_error_message('You must select a category of report')
-* add new rest endpoints
+return
+# everything's ok, move on to the next page
-but with great power comes great responsibility.
+self.client.template = 'add_page2'
-Interfaces.py has been around since the earliest releases of roundup
+def init(instance):
-and used to be the main way to get a lot of customization done. In
+instance.registerAction('page1_submit', Page1SubmitAction)
-modern roundup, the extensions_ mechanism is used, but there are
-places where interfaces.py is still useful. Note that the tracker
+4. Use the usual "new" action as the ``@action`` on the final page, and
-directories are not on the Python system path when interfaces.py is
+you're done (the standard context/submit method can do this for you).
-evaluated. You need to add library directories explictly by
-modifying sys.path.
+Silent Submit
-Example: Changing Cache-Control headers
+~~~~~~~~~~~~~
----------------------------------------
+When working on an issue, most of the time the people on the nosy list
+need to be notified of changes. There are cases where a user wants to
+add a comment to an issue and not bother other users on the nosy
+list.
+This feature is called Silent Submit because it allows the user to
+silently modify an issue and not tell anyone.
+There are several parts to this change. The main activity part
+involves editing the stock detectors/nosyreaction.py file in your
+tracker. Insert the following lines near the top of the nosyreaction
+function::
+# Did user click button to do a silent change?
+try:
+if db.web['submit'] == "silent_change":
+return
+except (AttributeError, KeyError) as err:
+# The web attribute or submit key don't exist.
+# That's fine. We were probably triggered by an email
+# or cli based change.
+pass
+This checks the submit button to see if it is the silent type. If there
+are exceptions trying to make that determination they are ignored and
+processing continues. You may wonder how db.web gets set. This is done
+by creating an extension. Add the file extensions/edit.py with
+this content::
+from roundup.cgi.actions import EditItemAction
+class Edit2Action(EditItemAction):
+def handle(self):
+self.db.web = {}  # create the dict
+# populate the dict by getting the value of the submit_button
+# element from the form.
+self.db.web['submit'] = self.form['submit_button'].value
+# call the core EditItemAction to process the edit.
+EditItemAction.handle(self)
+def init(instance):
+'''Override the default edit action with this new version'''
+instance.registerAction('edit', Edit2Action)
+This code is a wrapper for the Roundup EditItemAction. It checks the
+form's submit button to save the value element. The rest of the changes
+needed for the Silent Submit feature involves editing
+html/issue.item.html to add the silent submit button. In
+the stock issue.item.html the submit button is on a line that contains
+"submit button". Replace that line with something like the following::
+	    <input type="submit" name="submit_button"
+		   tal:condition="context/is_edit_ok"
+		   value="Submit Changes">&nbsp;
+	    <button type="submit" name="submit_button"
+		    tal:condition="context/is_edit_ok"
+		    title="Click this to submit but not send nosy email."
+		    value="silent_change" i18n:translate="">
+	      Silent Change</button>
+Note the difference in the value attribute for the two submit buttons.
+The value "silent_change" in the button specification must match the
+string in the nosy reaction function.
+Changing How the Core Code Works
+--------------------------------
+Changing Cache-Control Headers
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 The Client class in cgi/client.py has a lookup table that is used to
 set the Cache-Control headers for static files. The entries in this
 table are set from interfaces.py using::
 In this case static files delivered using @@file will have cache
 headers set. These files are searched for along the `static_files`
 path in the tracker's `config.ini`. In the example above:
 * a css file (e.g. @@file/style.css) will be cached for an hour
 * javascript files (e.g. @@file/libraries/jquery.js) will be cached
 for 30 seconds
 * a file named rss.xml will be cached for 15 minutes
 * a file named local.js will be cached for 2 hours
 Note that a file name match overrides the mime type settings.
-Example: Implement password complexity checking
+Implement Password Complexity Checking
------------------------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 .. index:: tracker; lib directory (example)
 This example uses the zxcvbn_ module that you can place in the zxcvbn
 subdirectory of your tracker's lib directory.
 it replaces the setPassword method in the Password class. The new
 version validates that the password is sufficiently complex. Then it
 passes off the setting of password to the original method.
-Example: Enhance time intervals
+Enhance Time Intervals Forms
--------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 To make the user interface easier to use, you may want to support
 other forms for intervals. For example you can support an interval
 like 1.5 by interpreting it the same as 1:30 (1 hour 30 minutes).
 Also you can allow a bare integer (e.g. 45) as a number of minutes.
 hyperdb.Interval.from_raw = normalizeperiod
 any call to convert an interval from raw form now has two simpler
 (and more friendly) ways to specify common time intervals.
-Example: Modifying the mail gateway
+Modifying the Mail Gateway
------------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~
 One site receives email on a main gateway. The virtual alias delivery
 table on the postfix server is configured with::
 test-issues@example.com  roundup-test@roundup-vm.example.com
 test-support@example.com roundup-test+support-a@roundup-vm.example.com
 test@support.example.com roundup-test+support-b@roundup-vm.example.com
-These modifications to the mail gateway for roundup allows anonymous
+These modifications to the mail gateway for Roundup allows anonymous
 submissions. It hides all of the requesters under the "support"
 user. It also makes some other modifications to the mail parser
 allowing keywords to be set and prefixes to be defined based on the
 delivery alias.
 	    else:
 		# parse the message normally
 		return roundup.mailgw.parsedMessage(mailgw, message)
 This is the most complex example section. The mail gateway is also one
-of the more complex subsystems in roundup, and modifying it is not
+of the more complex subsystems in Roundup, and modifying it is not
 trivial.
 Other Examples
---------------
+==============
 See the `rest interface documentation`_ for instructions on how to add
-new rest endpoints using interfaces.py.
+new rest endpoints or `change the rate limiting method`_  using interfaces.py.
-Database Content
+The `reference document`_ also has examples:
-================
+* `Extending the configuration file
-.. note::
+<reference.html#extending-the-configuration-file>`_.
-If you modify the content of definitional classes, you'll most
+* `Adding a new Permission <reference.html#adding-a-new-permission>`_
-likely need to edit the tracker `detectors`_ to reflect your changes.
+Examples on the Wiki
-Customisation of the special "definitional" classes (eg. status,
+====================
-priority, resolution, ...) may be done either before or after the
-tracker is initialised. The actual method of doing so is completely
+Even more examples of customization have been contributed by
-different in each case though, so be careful to use the right one.
+users. They can be found on the `wiki
+<https://wiki.roundup-tracker.org/CustomisationExamples>`_.
-**Changing content before tracker initialisation**
-Edit the initial_data.py module in your tracker to alter the items
-created using the ``create( ... )`` methods.
-**Changing content after tracker initialisation**
-As the "admin" user, click on the "class list" link in the web
-interface to bring up a list of all database classes. Click on the
-name of the class you wish to change the content of.
-You may also use the ``roundup-admin`` interface's create, set and
-retire methods to add, alter or remove items from the classes in
-question.
-See "`adding a new field to the classic schema`_" for an example that
-requires database content changes.
-Security / Access Controls
-==========================
-A set of Permissions is built into the security module by default:
-- Create (everything)
-- Edit (everything)
-- Search (everything) (used if View does not permit access)
-- View (everything)
-- Register (User class only)
-These are assigned to the "Admin" Role by default, and allow a user to do
-anything. Every Class you define in your `tracker schema`_ also gets an
-Create, Edit and View Permission of its own. The web and email interfaces
-also define:
-*Email Access*
-If defined, the user may use the email interface. Used by default to deny
-Anonymous users access to the email interface. When granted to the
-Anonymous user, they will be automatically registered by the email
-interface (see also the ``new_email_user_roles`` configuration option).
-*Web Access*
-If defined, the user may use the web interface. All users are able to see
-the login form, regardless of this setting (thus enabling logging in).
-*Web Roles*
-Controls user access to editing the "roles" property of the "user" class.
-TODO: deprecate in favour of a property-based control.
-*Rest Access* and *Xmlrpc Access*
-These control access to the Rest and Xmlrpc endpoints. The Admin and User
-roles have these by default in the classic tracker. See the
-`directions in the rest interface documentation`_ and the
-`xmlrpc interface documentation`_.
-These are hooked into the default Roles:
-- Admin (Create, Edit, Search, View and everything; Web Roles)
-- User (Web Access; Email Access)
-- Anonymous (Web Access)
-And finally, the "admin" user gets the "Admin" Role, and the "anonymous"
-user gets "Anonymous" assigned when the tracker is installed.
-For the "User" Role, the "classic" tracker defines:
-- Create, Edit and View issue, file, msg, query, keyword
-- View priority, status
-- View user
-- Edit their own user record
-And the "Anonymous" Role is defined as:
-- Web interface access
-- Register user (for registration)
-- View issue, file, msg, query, keyword, priority, status
-Put together, these settings appear in the tracker's ``schema.py`` file::
-#
-# TRACKER SECURITY SETTINGS
-#
-# See the configuration and customisation document for information
-# about security setup.
-#
-# REGULAR USERS
-#
-# Give the regular users access to the web and email interface
-db.security.addPermissionToRole('User', 'Web Access')
-db.security.addPermissionToRole('User', 'Email Access')
-db.security.addPermissionToRole('User', 'Rest Access')
-db.security.addPermissionToRole('User', 'Xmlrpc Access')
-# Assign the access and edit Permissions for issue, file and message
-# to regular users now
-for cl in 'issue', 'file', 'msg', 'keyword':
-db.security.addPermissionToRole('User', 'View', cl)
-db.security.addPermissionToRole('User', 'Edit', cl)
-db.security.addPermissionToRole('User', 'Create', cl)
-for cl in 'priority', 'status':
-db.security.addPermissionToRole('User', 'View', cl)
-# May users view other user information? Comment these lines out
-# if you don't want them to
-p = db.security.addPermission(name='View', klass='user',
-	properties=('id', 'organisation', 'phone', 'realname', 'timezone',
-	'username'))
-db.security.addPermissionToRole('User', p)
-# Users should be able to edit their own details -- this permission is
-# limited to only the situation where the Viewed or Edited item is their own.
-def own_record(db, userid, itemid, **ctx):
-'''Determine whether the userid matches the item being accessed.'''
-return userid == itemid
-p = db.security.addPermission(name='View', klass='user', check=own_record,
-description="User is allowed to view their own user details")
-db.security.addPermissionToRole('User', p)
-p = db.security.addPermission(name='Edit', klass='user', check=own_record,
-properties=('username', 'password', 'address', 'realname', 'phone',
-'organisation', 'alternate_addresses', 'queries', 'timezone'),
-description="User is allowed to edit their own user details")
-db.security.addPermissionToRole('User', p)
-# Users should be able to edit and view their own queries. They should also
-# be able to view any marked as not private. They should not be able to
-# edit others' queries, even if they're not private
-def view_query(db, userid, itemid):
-	private_for = db.query.get(itemid, 'private_for')
-	if not private_for: return True
-	return userid == private_for
-def edit_query(db, userid, itemid):
-	return userid == db.query.get(itemid, 'creator')
-p = db.security.addPermission(name='View', klass='query', check=view_query,
-	description="User is allowed to view their own and public queries")
-db.security.addPermissionToRole('User', p)
-p = db.security.addPermission(name='Search', klass='query')
-db.security.addPermissionToRole('User', p)
-p = db.security.addPermission(name='Edit', klass='query', check=edit_query,
-	description="User is allowed to edit their queries")
-db.security.addPermissionToRole('User', p)
-p = db.security.addPermission(name='Retire', klass='query', check=edit_query,
-	description="User is allowed to retire their queries")
-db.security.addPermissionToRole('User', p)
-p = db.security.addPermission(name='Restore', klass='query', check=edit_query,
-	description="User is allowed to restore their queries")
-db.security.addPermissionToRole('User', p)
-p = db.security.addPermission(name='Create', klass='query',
-	description="User is allowed to create queries")
-db.security.addPermissionToRole('User', p)
-#
-# ANONYMOUS USER PERMISSIONS
-#
-# Let anonymous users access the web interface. Note that almost all
-# trackers will need this Permission. The only situation where it's not
-# required is in a tracker that uses an HTTP Basic Authenticated front-end.
-db.security.addPermissionToRole('Anonymous', 'Web Access')
-# Let anonymous users access the email interface (note that this implies
-# that they will be registered automatically, hence they will need the
-# "Create" user Permission below)
-# This is disabled by default to stop spam from auto-registering users on
-# public trackers.
-#db.security.addPermissionToRole('Anonymous', 'Email Access')
-# Assign the appropriate permissions to the anonymous user's Anonymous
-# Role. Choices here are:
-# - Allow anonymous users to register
-db.security.addPermissionToRole('Anonymous', 'Register', 'user')
-# Allow anonymous users access to view issues (and the related, linked
-# information)
-for cl in 'issue', 'file', 'msg', 'keyword', 'priority', 'status':
-db.security.addPermissionToRole('Anonymous', 'View', cl)
-# Allow the anonymous user to use the "Show Unassigned" search.
-# It acts like "Show Open" if this permission is not available.
-# If you are running a tracker that does not allow read access for
-# anonymous, you should remove this entry as it can be used to perform
-# a username guessing attack against a roundup install.
-p = db.security.addPermission(name='Search', klass='user')
-db.security.addPermissionToRole ('Anonymous', p)
-# [OPTIONAL]
-# Allow anonymous users access to create or edit "issue" items (and the
-# related file and message items)
-#for cl in 'issue', 'file', 'msg':
-#   db.security.addPermissionToRole('Anonymous', 'Create', cl)
-#   db.security.addPermissionToRole('Anonymous', 'Edit', cl)
-.. index::
-single: roundup-admin; view class permissions
-You can use ``roundup-admin security`` to verify the permissions
-defined in the schema. It also verifies that properties specified in
-permissions are valid for the class. This helps detect typos that can
-cause baffling permission issues.
-Automatic Permission Checks
----------------------------
-Permissions are automatically checked when information is rendered
-through the web. This includes:
-1. View checks for properties when being rendered via the ``plain()`` or
-similar methods. If the check fails, the text "[hidden]" will be
-displayed.
-2. Edit checks for properties when the edit field is being rendered via
-the ``field()`` or similar methods. If the check fails, the property
-will be rendered via the ``plain()`` method (see point 1. for subsequent
-checking performed)
-3. View checks are performed in index pages for each item being displayed
-such that if the user does not have permission, the row is not rendered.
-4. View checks are performed at the top of item pages for the Item being
-displayed. If the user does not have permission, the text "You are not
-allowed to view this page." will be displayed.
-5. View checks are performed at the top of index pages for the Class being
-displayed. If the user does not have permission, the text "You are not
-allowed to view this page." will be displayed.
-New User Roles
---------------
-New users are assigned the Roles defined in the config file as:
-- NEW_WEB_USER_ROLES
-- NEW_EMAIL_USER_ROLES
-The `users may only edit their issues`_ example shows customisation of
-these parameters.
-Changing Access Controls
-------------------------
-You may alter the configuration variables to change the Role that new
-web or email users get, for example to not give them access to the web
-interface if they register through email.
-You may use the ``roundup-admin`` "``security``" command to display the
-current Role and Permission configuration in your tracker.
-Adding a new Permission
-~~~~~~~~~~~~~~~~~~~~~~~
-When adding a new Permission, you will need to:
-1. add it to your tracker's ``schema.py`` so it is created, using
-``security.addPermission``, for example::
-db.security.addPermission(name="View", klass='frozzle',
-description="User is allowed to access frozzles")
-will set up a new "View" permission on the Class "frozzle".
-2. enable it for the Roles that should have it (verify with
-"``roundup-admin security``")
-3. add it to the relevant HTML interface templates
-4. add it to the appropriate xxxPermission methods on in your tracker
-interfaces module
-The ``addPermission`` method takes a few optional parameters:
-**properties**
-A sequence of property names that are the only properties to apply the
-new Permission to (eg. ``... klass='user', properties=('name',
-'email') ...``)
-**props_only**
-A boolean value (set to false by default) that is a new feature
-in roundup 1.6.
-A permission defined using:
-``properties=('list', 'of', 'property', 'names')``
-is used to determine access for things other than just those
-properties. For example a check for View permission on the issue
-class or an issue item can use any View permission for the issue
-class even if that permission has a property list.  This can be
-confusing and surprising as you would think that a permission
-including properties would be used only for determining the
-access permission for those properties.
-``roundup-admin security`` will report invalid properties for the
-class. For example a permission with an invalid summary property is
-presented as::
-Allowed to see content of object regardless of spam status
-(View for "file": ('content', 'summary') only)
-**Invalid properties for file: ['summary']
-Setting ``props_only=True`` will make the permission valid only for
-those properties.
-If you use a lot of permissions with property checks, it can be
-difficult to change all of them. Calling the function:
-db.security.set_props_only_default(True)
-at the top of ``schema.py`` will make every permission creation
-behave as though props_only was set to True. It is expected that the
-default of True will become the default in a future Roundup release.
-**check**
-A function to be executed which returns boolean determining whether
-the Permission is allowed. If it returns True, the permission is
-allowed, if it returns False the permission is denied.  The function
-can have one of two signatures::
-check(db, userid, itemid)
-or::
-check(db, userid, itemid, **ctx)
-where ``db`` is a handle on the open database, ``userid`` is
-the user attempting access and ``itemid`` is the specific item being
-accessed. If the second form is used the ``ctx`` dictionary is
-defined with the following values::
-ctx['property'] the name of the property being checked or None if
-it's a class check.
-ctx['classname'] the name of the class that is being checked
-(issue, query ....).
-ctx['permission'] the name of the permission (e.g. View, Edit...).
-The second form is preferred as it makes it easier to implement more
-complex permission schemes. An example of the use of ``ctx`` can be
-found in the ``upgrading.txt`` or `upgrading.html`_ document.
-.. _`upgrading.html`: upgrading.html
-Example Scenarios
-~~~~~~~~~~~~~~~~~
-See the `examples`_ section for longer examples of customisation.
-**anonymous access through the e-mail gateway**
-Give the "anonymous" user the "Email Access", ("Edit", "issue") and
-("Create", "msg") Permissions but do not not give them the ("Create",
-"user") Permission. This means that when an unknown user sends email
-into the tracker, they're automatically logged in as "anonymous".
-Since they don't have the ("Create", "user") Permission, they won't
-be automatically registered, but since "anonymous" has permission to
-use the gateway, they'll still be able to submit issues. Note that
-the Sender information - their email address - will not be available
-- they're *anonymous*.
-**automatic registration of users in the e-mail gateway**
-By giving the "anonymous" user the ("Register", "user") Permission, any
-unidentified user will automatically be registered with the tracker
-(with no password, so they won't be able to log in through
-the web until an admin sets their password). By default new Roundup
-trackers don't allow this as it opens them up to spam. It may be enabled
-by uncommenting the appropriate addPermissionToRole in your tracker's
-``schema.py`` file. The new user is given the Roles list defined in the
-"new_email_user_roles" config variable.
-**only developers may be assigned issues**
-Create a new Permission called "Fixer" for the "issue" class. Create a
-new Role "Developer" which has that Permission, and assign that to the
-appropriate users. Filter the list of users available in the assignedto
-list to include only those users. Enforce the Permission with an
-auditor. See the example
-`restricting the list of users that are assignable to a task`_.
-**only managers may sign off issues as complete**
-Create a new Permission called "Closer" for the "issue" class. Create a
-new Role "Manager" which has that Permission, and assign that to the
-appropriate users. In your web interface, only display the "resolved"
-issue state option when the user has the "Closer" Permissions. Enforce
-the Permission with an auditor. This is very similar to the previous
-example, except that the web interface check would look like::
-<option tal:condition="python:request.user.hasPermission('Closer')"
-value="resolved">Resolved</option>
-**don't give web access to users who register through email**
-Create a new Role called "Email User" which has all the Permissions of
-the normal "User" Role minus the "Web Access" Permission. This will
-allow users to send in emails to the tracker, but not access the web
-interface.
-**let some users edit the details of all users**
-Create a new Role called "User Admin" which has the Permission for
-editing users::
-db.security.addRole(name='User Admin', description='Managing users')
-p = db.security.getPermission('Edit', 'user')
-db.security.addPermissionToRole('User Admin', p)
-and assign the Role to the users who need the permission.
-Web Interface
-=============
-.. contents::
-:local:
-The web interface is provided by the ``roundup.cgi.client`` module and
-is used by ``roundup.cgi``, ``roundup-server`` and ``ZRoundup``
-(``ZRoundup``  is broken, until further notice). In all cases, we
-determine which tracker is being accessed (the first part of the URL
-path inside the scope of the CGI handler) and pass control on to the
-``roundup.cgi.client.Client`` class - which handles the rest of the
-access through its ``main()`` method. This means that you can do pretty
-much anything you want as a web interface to your tracker.
-Repercussions of changing the tracker schema
----------------------------------------------
-If you choose to change the `tracker schema`_ you will need to ensure
-the web interface knows about it:
-1. Index, item and search pages for the relevant classes may need to
-have properties added or removed,
-2. The "page" template may require links to be changed, as might the
-"home" page's content arguments.
-How requests are processed
---------------------------
-The basic processing of a web request proceeds as follows:
-1. figure out who we are, defaulting to the "anonymous" user
-2. figure out what the request is for - we call this the "context"
-3. handle any requested action (item edit, search, ...)
-4. render the template requested by the context, resulting in HTML
-output
-In some situations, exceptions occur:
-- HTTP Redirect  (generally raised by an action)
-- SendFile       (generally raised by ``determine_context``)
-here we serve up a FileClass "content" property
-- SendStaticFile (generally raised by ``determine_context``)
-here we serve up a file from the tracker "html" directory
-- Unauthorised   (generally raised by an action)
-here the action is cancelled, the request is rendered and an error
-message is displayed indicating that permission was not granted for
-the action to take place
-- NotFound       (raised wherever it needs to be)
-this exception percolates up to the CGI interface that called the
-client
-Roundup URL design
-------------------
-Each tracker has several hardcoded URLs. These three are equivalent and
-lead to the main tracker page:
-1. ``/``
-2. ``/index``
-3. ``/home``
-The following prefix is used to access static resources:
-4. ``/@@file/``
-Two additional url's are used for the API's.
-The `REST api`_ is accessed via:
-5. ``/rest/``
-.. _`REST api`: rest.html
-and the `XMLRPC api`_ is available at:
-6. ``/rest/``
-.. _`XMLRPC api`: xmlrpc.html
-All other URLs depend on the classes configured in Roundup database.
-Each class receives two URLs - one for the class itself and another
-for specific items of that class. Example for class URL:
-7. ``/issue``
-This is usually used to show listings of class items. The URL for
-for specific object of issue class with id 1 will look like:
-8. ``/issue1``
-.. _strip_zeros:
-Note that a leading string of 0's will be stripped from the id part of
-the object designator in the URL. E.G. ``/issue001`` is the same as
-``/issue1``.  Similarly for ``/file01`` etc. However you should
-generate URL's without the extra zeros.
-Determining web context
------------------------
-To determine the "context" of a request (what request is for), we look at
-the URL path after the tracker root and at ``@template`` request
-parameter. Typical URL paths look like:
-1.  ``/tracker/issue``
-2.  ``/tracker/issue1``
-3.  ``/tracker/@@file/style.css``
-4.  ``/cgi-bin/roundup.cgi/tracker/file1``
-5.  ``/cgi-bin/roundup.cgi/tracker/file1/kitten.png``
-where tracker root is ``/tracker/`` or ``/cgi-bin/roundup.cgi/tracker/``
-We're looking at "issue", "issue1", "@@file/style.css", "file1" and
-"file1/kitten.png" in the cases above.
-a. with is no path we are in the "home" context. See `the "home"
-context`_ below for details. "index" or "home" paths may also be used
-to switch into "home" context.
-b. for paths starting with "@@file" the additional path entry ("style.css"
-in the example above) specifies the static file to be served
-from the tracker TEMPLATES directory (or STATIC_FILES, if configured).
-This is usually the tracker's "html" directory. Internally this works
-by raising SendStaticFile exception.
-c. if there is something in the path (as in example 1, "issue"), it
-identifies the tracker class to display.
-d. if the path is an item designator (as in examples 2 and 4, "issue1"
-and "file1"), then we're to display a specific item.
-:ref:`Note. <strip_zeros>`
-e. if the path starts with an item designator and is longer than one
-entry (as in example 5, "file1/kitten.png"), then we're assumed to be
-handling an item of a ``FileClass``, and the extra path information
-gives the filename that the client is going to label the download
-with (i.e. "file1/kitten.png" is nicer to download than "file1").
-This raises a ``SendFile`` exception.
-Neither b. or e. use templates and stop before the template is
-determined. For other contexts the template used is specified by the
-``@template`` variable, which defaults to:
-- only classname supplied:       "index"
-- full item designator supplied: "item"
-The "home" Context
-------------------
-The "home" context is special because it allows you to add templated
-pages to your tracker that don't rely on a class or item (ie. an issues
-list or specific issue).
-Let's say you wish to add frames to control the layout of your tracker's
-interface. You'd probably have:
-- A top-level frameset page. This page probably wouldn't be templated, so
-it could be served as a static file (see `serving static content`_)
-- A sidebar frame that is templated. Let's call this page
-"home.navigation.html" in your tracker's "html" directory. To load that
-page up, you use the URL:
-<tracker url>/home?@template=navigation
-Serving static content
-----------------------
-See the previous section `determining web context`_ where it describes
-``@@file`` paths.
-Performing actions in web requests
-----------------------------------
-When a user requests a web page, they may optionally also request for an
-action to take place. As described in `how requests are processed`_, the
-action is performed before the requested page is generated. Actions are
-triggered by using a ``@action`` CGI variable, where the value is one
-of:
-**login**
-Attempt to log a user in.
-**logout**
-Log the user out - make them "anonymous".
-**register**
-Attempt to create a new user based on the contents of the form and then
-log them in.
-**edit**
-Perform an edit of an item in the database. There are some `special form
-variables`_ you may use. Also you can set the ``__redirect_to`` form
-variable to the URL that should be displayed after the edit is succesfully
-completed. If you wanted to edit a sequence of issues, users etc. this
-could be used to display the next item in the sequence to the user.
-**new**
-Add a new item to the database. You may use the same `special form
-variables`_ as in the "edit" action. Also you can set the
-``__redirect_to`` form variable to the URL that should be displayed after
-the new item is created. This is useful if you want to create another
-item rather than edit the newly created item.
-**retire**
-Retire the item in the database.
-**editCSV**
-Performs an edit of all of a class' items in one go. See also the
-*class*.csv templating method which generates the CSV data to be
-edited, and the ``'_generic.index'`` template which uses both of these
-features.
-**search**
-Mangle some of the form variables:
-- Set the form ":filter" variable based on the values of the filter
-variables - if they're set to anything other than "dontcare" then add
-them to :filter.
-- Also handle the ":queryname" variable and save off the query to the
-user's query list.
-Each of the actions is implemented by a corresponding ``*XxxAction*`` (where
-"Xxx" is the name of the action) class in the ``roundup.cgi.actions`` module.
-These classes are registered with ``roundup.cgi.client.Client``. If you need
-to define new actions, you may add them there (see `defining new
-web actions`_).
-Each action class also has a ``*permission*`` method which determines whether
-the action is permissible given the current user. The base permission checks
-for each action are:
-**login**
-Determine whether the user has the "Web Access" Permission.
-**logout**
-No permission checks are made.
-**register**
-Determine whether the user has the ("Create", "user") Permission.
-**edit**
-Determine whether the user has permission to edit this item. If we're
-editing the "user" class, users are allowed to edit their own details -
-unless they try to edit the "roles" property, which requires the
-special Permission "Web Roles".
-**new**
-Determine whether the user has permission to create this item. No
-additional property checks are made. Additionally, new user items may
-be created if the user has the ("Create", "user") Permission.
-**editCSV**
-Determine whether the user has permission to edit this class.
-**search**
-Determine whether the user has permission to view this class.
-Protecting users from web application attacks
----------------------------------------------
-There is a class of attacks known as Cross Site Request Forgeries
-(CSRF). Malicious code running in the browser can making a
-request to roundup while you are logged into roundup.  The
-malicious code piggy backs on your existing roundup session to
-make changes without your knowledge. Roundup 1.6 has support for
-defending against this by analyzing the
-* Referer,
-* Origin, and
-* Host or
-* X-Forwarded-Host
-HTTP headers. It compares the headers to the value of the web setting
-in the [tracker] section of the tracker's ``config.ini``.
-Also a per form token (also called a nonce) can be enabled for
-the tracker using the ``csrf_enforce_token`` option in
-config.ini.  When enabled, roundup will validate a hidden form
-field called ``@csrf``. If the validation fails (or the token
-is used more than once) the request is rejected.  The ``@csrf``
-input field is added automatically by calling the ``submit``
-function/path. It can also be added manually by calling
-anti_csrf_nonce() directly. For example:
-<input name="@csrf" type="hidden"
-tal:attributes="value python:utils.anti_csrf_nonce(lifetime=10)">
-By default a nonce lifetime is 2 weeks. However the lifetime (in
-minutes) can be set by passing a lifetime argument as shown
-above. The example above makes the nonce lifetime 10 minutes.
-Search for @csrf in this document for more examples. There are
-more examples and information in ``upgrading.txt``.
-The token protects you because malicious code supplied by another
-site is unable to obtain the token. Thus many attempts they make
-to submit a request are rejected.
-The protection on the xmlrpc interface is untested, but is based
-on a valid header check against the roundup url and the presence
-of the ``X-REQUESTED-WITH`` header. Work to improve this is a
-future project after the 1.6 release.
-The enforcement levels can be modified in ``config.ini``. Refer to
-that file for details.
-Special form variables
-----------------------
-Item properties and their values are edited with html FORM
-variables and their values. You can:
-- Change the value of some property of the current item.
-- Create a new item of any class, and edit the new item's
-properties,
-- Attach newly created items to a multilink property of the
-current item.
-- Remove items from a multilink property of the current item.
-- Specify that some properties are required for the edit
-operation to be successful.
-- Redirect to a different page after creating a new item (new action
-only, not edit action). Usually you end up on the page for the
-created item.
-- Set up user interface locale.
-These operations will only take place if the form action (the
-``@action`` variable) is "edit" or "new".
-In the following, <bracketed> values are variable, "@" may be
-either ":" or "@", and other text "required" is fixed.
-Two special form variables are used to specify user language preferences:
-``@language``
-value may be locale name or ``none``. If this variable is set to
-locale name, web interface language is changed to given value
-(provided that appropriate translation is available), the value
-is stored in the browser cookie and will be used for all following
-requests.  If value is ``none`` the cookie is removed and the
-language is changed to the tracker default, set up in the tracker
-configuration or OS environment.
-``@charset``
-value may be character set name or ``none``.  Character set name
-is stored in the browser cookie and sets output encoding for all
-HTML pages generated by Roundup.  If value is ``none`` the cookie
-is removed and HTML output is reset to Roundup internal encoding
-(UTF-8).
-Most properties are specified as form variables:
-``<propname>``
-property on the current context item
-``<designator>"@"<propname>``
-property on the indicated item (for editing related information)
-Designators name a specific item of a class.
-``<classname><N>``
-Name an existing item of class <classname>.
-``<classname>"-"<N>``
-Name the <N>th new item of class <classname>. If the form
-submission is successful, a new item of <classname> is
-created. Within the submitted form, a particular
-designator of this form always refers to the same new
-item.
-Once we have determined the "propname", we look at it to see
-if it's special:
-``@required``
-The associated form value is a comma-separated list of
-property names that must be specified when the form is
-submitted for the edit operation to succeed.
-When the <designator> is missing, the properties are
-for the current context item.  When <designator> is
-present, they are for the item specified by
-<designator>.
-The "@required" specifier must come before any of the
-properties it refers to are assigned in the form.
-``@remove@<propname>=id(s)`` or ``@add@<propname>=id(s)``
-The "@add@" and "@remove@" edit actions apply only to
-Multilink properties.  The form value must be a
-comma-separate list of keys for the class specified by
-the simple form variable.  The listed items are added
-to (respectively, removed from) the specified
-property.
-``@link@<propname>=<designator>``
-If the edit action is "@link@", the simple form
-variable must specify a Link or Multilink property.
-The form value is a comma-separated list of
-designators.  The item corresponding to each
-designator is linked to the property given by simple
-form variable.
-None of the above (ie. just a simple form value)
-The value of the form variable is converted
-appropriately, depending on the type of the property.
-For a Link('klass') property, the form value is a
-single key for 'klass', where the key field is
-specified in schema.py.
-For a Multilink('klass') property, the form value is a
-comma-separated list of keys for 'klass', where the
-key field is specified in schema.py.
-Note that for simple-form-variables specifiying Link
-and Multilink properties, the linked-to class must
-have a key field.
-For a String() property specifying a filename, the
-file named by the form value is uploaded. This means we
-try to set additional properties "filename" and "type" (if
-they are valid for the class).  Otherwise, the property
-is set to the form value.
-For Date(), Interval(), Boolean(), Integer() and Number()
-properties, the form value is converted to the
-appropriate value.
-Any of the form variables may be prefixed with a classname or
-designator.
-Setting the form variable: ``__redirect_to=`` to a url when
-@action=new redirects the user to the specified url after successfully
-creating the new item. This is useful if you want the user to create
-another item rather than edit the newly created item.  Note that the
-url assigned to ``__redirect_to`` must be url encoded/quoted and be
-under the tracker's base url. If the base_url uses http, you can set
-the url to https.
-Two special form values are supported for backwards compatibility:
-@note
-This is equivalent to::
-@link@messages=msg-1
-msg-1@content=value
-which is equivalent to the html::
-<textarea name="msg-1@content"></textarea>
-<input type="hidden" name="@link@messages" value="msg-1">
-except that in addition, the "author" and "date" properties of
-"msg-1" are set to the userid of the submitter, and the current
-time, respectively.
-@file
-This is equivalent to::
-@link@files=file-1
-file-1@content=value
-by adding the HTML::
-<input type="file" name="file-1@content">
-<input type="hidden" name="@link@files" value="file-1">
-The String content value is handled as described above for file
-uploads.
-If both the "@note" and "@file" form variables are
-specified, the action::
-msg-1@link@files=file-1
-is also performed. This would be expressed in HTML with::
-<input type="hidden" name="msg-1@link@files" value="file-1">
-We also check that FileClass items have a "content" property with
-actual content, otherwise we remove them from all_props before
-returning.
-Default templates
------------------
-The default templates are html4 compliant. If you wish to change them to be
-xhtml compliant, you'll need to change the ``html_version`` configuration
-variable in ``config.ini`` to ``'xhtml'`` instead of ``'html4'``.
-Most customisation of the web view can be done by modifying the
-templates in the tracker ``'html'`` directory. There are several types
-of files in there. The *minimal* template includes:
-**page.html**
-This template usually defines the overall look of your tracker. When
-you view an issue, it appears inside this template. When you view an
-index, it also appears inside this template. This template defines a
-macro called "icing" which is used by almost all other templates as a
-coating for their content, using its "content" slot. It also defines
-the "head_title" and "body_title" slots to allow setting of the page
-title.
-**home.html**
-the default page displayed when no other page is indicated by the user
-**home.classlist.html**
-a special version of the default page that lists the classes in the
-tracker
-**classname.item.html**
-displays an item of the *classname* class
-**classname.index.html**
-displays a list of *classname* items
-**classname.search.html**
-displays a search page for *classname* items
-**_generic.index.html**
-used to display a list of items where there is no
-``*classname*.index`` available
-**_generic.help.html**
-used to display a "class help" page where there is no
-``*classname*.help``
-**user.register.html**
-a special page just for the user class, that renders the registration
-page
-**style.css**
-a static file that is served up as-is
-The *classic* template has a number of additional templates.
-Remember that you can create any template extension you want to,
-so if you just want to play around with the templating for new issues,
-you can copy the current "issue.item" template to "issue.test", and then
-access the test template using the "@template" URL argument::
-http://your.tracker.example/tracker/issue?@template=test
-and it won't affect your users using the "issue.item" template.
-You can also put templates into a subdirectory of the template
-directory. So if you specify::
-http://your.tracker.example/tracker/issue?@template=test/item
-you will use the template at: ``test/issue.item.html``. If that
-template doesn't exit it will try to use
-``test/_generic.item.html``. If that template doesn't exist
-it will return an error.
-Implementing Modal Editing Using @template
-------------------------------------------
-Many item templates allow you to edit the item. They contain
-code that renders edit boxes if the user has edit permissions.
-Otherwise the template will just display the item information.
-In some cases you want to do a modal edit. The user has to take some
-action (click a button or follow a link) to shift from display mode to
-edit mode. When the changes are submitted, ending the edit mode,
-the user is returned to display mode.
-Modal workflows usually slow things down and are not implemented by
-default templates. However for some workflows a modal edit is useful.
-For example a batch edit mode that allows the user to edit a number of
-issues all from one form could be implemented as a modal workflow of:
-* search for issues to modify
-* switch to edit mode and change values
-* exit back to the results of the search
-To implement the modal edit, assume you have an issue.edit.html
-template that implements an edit form.  On the display page (a version
-of issue.item.html modified to only display information) add a link
-that calls the display url, but adds ``@template=edit`` to the link.
-This will now display the edit page. On the edit page you want to add
-a hidden text field to your form named ``@template`` with the value:
-``item|edit``.  When the form is submitted it is validated. If the
-form is correct the user will see the item rendered using the item
-template. If there is an error (validation failed) the item will be
-rendered using the edit template. The edit template that is rendered
-will display all the changes that the user made to the form before it
-was submitted. The user can correct the error and resubmit the changes
-until the form validates.
-If the form failed to validate but the ``@template`` field had the
-value ``item`` the user would still see the error, but all of the data
-the user entered would be discarded. The user would have to redo all
-the edits again.
-How the templates work
-----------------------
-Templating engines
-~~~~~~~~~~~~~~~~~~
-Since version 1.4.20 Roundup supports two templating engines:
-* the original `Template Attribute Language`_ (TAL) engine from Zope
-* the standalone Chameleon templating engine. Chameleon is intended
-as a replacement for the original TAL engine, and supports the
-same syntax, but they are not 100% compatible. The major (and most
-likely the only) incompatibility is the default expression type being
-``python:`` instead of ``path:``. See also "Incompatibilities and
-differences" section of `Chameleon documentation`__.
-Version 1.5.0 added experimental support for the `jinja2`_ templating
-language. You must install the `jinja2`_ module in order to use it. The
-``jinja2`` template supplied with Roundup has the templates rewritten
-to use ``jinja2`` rather than TAL. A number of trackers are running
-using ``jinja2`` templating so it is considered less experimental than
-Chameleon templating.
-.. _jinja2: https://palletsprojects.com/p/jinja/
-**NOTE1**: For historical reasons, examples given below assumes path
-expression as default expression type. With Chameleon you have to manually
-resolve the path expressions. A Chameleon-based, z3c.pt, that is fully
-compatible with the old TAL implementation, is planned to be included in a
-future release.
-**NOTE2**: As of 1.4.20 Chameleon support is highly experimental and **not**
-recommended for production use.
-.. _Chameleon:
-https://pypi.org/project/Chameleon/
-.. _z3c.pt:
-https://pypi.org/project/z3c.pt/
-__ https://chameleon.readthedocs.io/en/latest/reference.html?highlight=differences#incompatibilities-and-differences
-.. _TAL:
-.. _Template Attribute Language:
-https://pagetemplates.readthedocs.io/en/latest/history/TALSpecification14.html
-Basic Templating Actions
-~~~~~~~~~~~~~~~~~~~~~~~~
-Roundup's templates consist of special attributes on the HTML tags.
-These attributes form the **Template Attribute Language**, or TAL.
-The basic TAL commands are:
-**tal:define="variable expression; variable expression; ..."**
-Define a new variable that is local to this tag and its contents. For
-example::
-<html tal:define="title request/description">
-<head><title tal:content="title"></title></head>
-</html>
-In this example, the variable "title" is defined as the result of the
-expression "request/description". The "tal:content" command inside the
-<html> tag may then use the "title" variable.
-**tal:condition="expression"**
-Only keep this tag and its contents if the expression is true. For
-example::
-<p tal:condition="python:request.user.hasPermission('View', 'issue')">
-Display some issue information.
-</p>
-In the example, the <p> tag and its contents are only displayed if
-the user has the "View" permission for issues. We consider the number
-zero, a blank string, an empty list, and the built-in variable
-nothing to be false values. Nearly every other value is true,
-including non-zero numbers, and strings with anything in them (even
-spaces!).
-**tal:repeat="variable expression"**
-Repeat this tag and its contents for each element of the sequence
-that the expression returns, defining a new local variable and a
-special "repeat" variable for each element. For example::
-<tr tal:repeat="u user/list">
-<td tal:content="u/id"></td>
-<td tal:content="u/username"></td>
-<td tal:content="u/realname"></td>
-</tr>
-The example would iterate over the sequence of users returned by
-"user/list" and define the local variable "u" for each entry. Using
-the repeat command creates a new variable called "repeat" which you
-may access to gather information about the iteration. See the section
-below on `the repeat variable`_.
-**tal:replace="expression"**
-Replace this tag with the result of the expression. For example::
-<span tal:replace="request/user/realname" />
-The example would replace the <span> tag and its contents with the
-user's realname. If the user's realname was "Bruce", then the
-resultant output would be "Bruce".
-**tal:content="expression"**
-Replace the contents of this tag with the result of the expression.
-For example::
-<span tal:content="request/user/realname">user's name appears here
-</span>
-The example would replace the contents of the <span> tag with the
-user's realname. If the user's realname was "Bruce" then the
-resultant output would be "<span>Bruce</span>".
-**tal:attributes="attribute expression; attribute expression; ..."**
-Set attributes on this tag to the results of expressions. For
-example::
-<a tal:attributes="href string:user${request/user/id}">My Details</a>
-In the example, the "href" attribute of the <a> tag is set to the
-value of the "string:user${request/user/id}" expression, which will
-be something like "user123".
-**tal:omit-tag="expression"**
-Remove this tag (but not its contents) if the expression is true. For
-example::
-<span tal:omit-tag="python:1">Hello, world!</span>
-would result in output of::
-Hello, world!
-Note that the commands on a given tag are evaulated in the order above,
-so *define* comes before *condition*, and so on.
-Additionally, you may include tags such as <tal:block>, which are
-removed from output. Its content is kept, but the tag itself is not (so
-don't go using any "tal:attributes" commands on it). This is useful for
-making arbitrary blocks of HTML conditional or repeatable (very handy
-for repeating multiple table rows, which would otherwise require an
-illegal tag placement to effect the repeat).
-Templating Expressions
-~~~~~~~~~~~~~~~~~~~~~~
-Templating Expressions are covered by `Template Attribute Language
-Expression Syntax`_, or TALES. The expressions you may use in the
-attribute values may be one of the following forms:
-**Path Expressions** - eg. ``item/status/checklist``
-These are object attribute / item accesses. Roughly speaking, the
-path ``item/status/checklist`` is broken into parts ``item``,
-``status`` and ``checklist``. The ``item`` part is the root of the
-expression. We then look for a ``status`` attribute on ``item``, or
-failing that, a ``status`` item (as in ``item['status']``). If that
-fails, the path expression fails. When we get to the end, the object
-we're left with is evaluated to get a string - if it is a method, it
-is called; if it is an object, it is stringified. Path expressions
-may have an optional ``path:`` prefix, but they are the default
-expression type, so it's not necessary.
-If an expression evaluates to ``default``, then the expression is
-"cancelled" - whatever HTML already exists in the template will
-remain (tag content in the case of ``tal:content``, attributes in the
-case of ``tal:attributes``).
-If an expression evaluates to ``nothing`` then the target of the
-expression is removed (tag content in the case of ``tal:content``,
-attributes in the case of ``tal:attributes`` and the tag itself in
-the case of ``tal:replace``).
-If an element in the path may not exist, then you can use the ``|``
-operator in the expression to provide an alternative. So, the
-expression ``request/form/foo/value | default`` would simply leave
-the current HTML in place if the "foo" form variable doesn't exist.
-You may use the python function ``path``, as in
-``path("item/status")``, to embed path expressions in Python
-expressions.
-**String Expressions** - eg. ``string:hello ${user/name}``
-These expressions are simple string interpolations - though they can
-be just plain strings with no interpolation if you want. The
-expression in the ``${ ... }`` is just a path expression as above.
-**Python Expressions** - eg. ``python: 1+1``
-These expressions give the full power of Python. All the "root level"
-variables are available, so ``python:item.status.checklist()`` would
-be equivalent to ``item/status/checklist``, assuming that
-``checklist`` is a method.
-Modifiers:
-**structure** - eg. ``structure python:msg.content.plain(hyperlink=1)``
-The result of expressions are normally *escaped* to be safe for HTML
-display (all "<", ">" and "&" are turned into special entities). The
-``structure`` expression modifier turns off this escaping - the
-result of the expression is now assumed to be HTML, which is passed
-to the web browser for rendering.
-**not:** - eg. ``not:python:1=1``
-This simply inverts the logical true/false value of another
-expression.
-.. _TALES:
-.. _Template Attribute Language Expression Syntax:
-https://pagetemplates.readthedocs.io/en/latest/history/TALESSpecification13.html
-Template Macros
-~~~~~~~~~~~~~~~
-Macros are used in Roundup to save us from repeating the same common
-page stuctures over and over. The most common (and probably only) macro
-you'll use is the "icing" macro defined in the "page" template.
-Macros are generated and used inside your templates using special
-attributes similar to the `basic templating actions`_. In this case,
-though, the attributes belong to the `Macro Expansion Template
-Attribute Language`_, or METAL. The macro commands are:
-**metal:define-macro="macro name"**
-Define that the tag and its contents are now a macro that may be
-inserted into other templates using the *use-macro* command. For
-example::
-<html metal:define-macro="page">
-...
-</html>
-defines a macro called "page" using the ``<html>`` tag and its
-contents. Once defined, macros are stored on the template they're
-defined on in the ``macros`` attribute. You can access them later on
-through the ``templates`` variable, eg. the most common
-``templates/page/macros/icing`` to access the "page" macro of the
-"page" template.
-**metal:use-macro="path expression"**
-Use a macro, which is identified by the path expression (see above).
-This will replace the current tag with the identified macro contents.
-For example::
-<tal:block metal:use-macro="templates/page/macros/icing">
-...
-</tal:block>
-will replace the tag and its contents with the "page" macro of the
-"page" template.
-**metal:define-slot="slot name"** and **metal:fill-slot="slot name"**
-To define *dynamic* parts of the macro, you define "slots" which may
-be filled when the macro is used with a *use-macro* command. For
-example, the ``templates/page/macros/icing`` macro defines a slot like
-so::
-<title metal:define-slot="head_title">title goes here</title>
-In your *use-macro* command, you may now use a *fill-slot* command
-like this::
-<title metal:fill-slot="head_title">My Title</title>
-where the tag that fills the slot completely replaces the one defined
-as the slot in the macro.
-Note that you may not mix `METAL`_ and `TAL`_ commands on the same tag, but
-TAL commands may be used freely inside METAL-using tags (so your
-*fill-slots* tags may have all manner of TAL inside them).
-.. _METAL:
-.. _Macro Expansion Template Attribute Language:
-https://pagetemplates.readthedocs.io/en/latest/history/TALESSpecification13.html
-Information available to templates
-----------------------------------
-This is implemented by ``roundup.cgi.templating.RoundupPageTemplate``
-The following variables are available to templates.
-**context**
-The current context. This is either None, a `hyperdb class wrapper`_
-or a `hyperdb item wrapper`_
-**request**
-Includes information about the current request, including:
-- the current index information (``filterspec``, ``filter`` args,
-``properties``, etc) parsed out of the form.
-- methods for easy filterspec link generation
-- "form"
-The current CGI form information as a mapping of form argument name
-to value (specifically a cgi.FieldStorage)
-- "env" the CGI environment variables
-- "base" the base URL for this instance
-- "user" a HTMLItem instance for the current user
-- "language" as determined by the browser or config
-- "classname" the current classname (possibly None)
-- "template" the current template (suffix, also possibly None)
-**config**
-This variable holds all the values defined in the tracker config.ini
-file (eg. TRACKER_NAME, etc.)
-**db**
-The current database, used to access arbitrary database items.
-**templates**
-Access to all the tracker templates by name. Used mainly in
-*use-macro* commands.
-**utils**
-This variable makes available some utility functions like batching.
-**nothing**
-This is a special variable - if an expression evaluates to this, then
-the tag (in the case of a ``tal:replace``), its contents (in the case
-of ``tal:content``) or some attributes (in the case of
-``tal:attributes``) will not appear in the the output. So, for
-example::
-<span tal:attributes="class nothing">Hello, World!</span>
-would result in::
-<span>Hello, World!</span>
-**default**
-Also a special variable - if an expression evaluates to this, then the
-existing HTML in the template will not be replaced or removed, it will
-remain. So::
-<span tal:replace="default">Hello, World!</span>
-would result in::
-<span>Hello, World!</span>
-**true**, **false**
-Boolean constants that may be used in `templating expressions`_
-instead of ``python:1`` and ``python:0``.
-**i18n**
-Internationalization service, providing two string translation methods:
-**gettext** (*message*)
-Return the localized translation of message
-**ngettext** (*singular*, *plural*, *number*)
-Like ``gettext()``, but consider plural forms. If a translation
-is found, apply the plural formula to *number*, and return the
-resulting message (some languages have more than two plural forms).
-If no translation is found, return singular if *number* is 1;
-return plural otherwise.
-The context variable
-~~~~~~~~~~~~~~~~~~~~
-The *context* variable is one of three things based on the current
-context (see `determining web context`_ for how we figure this out):
-1. if we're looking at a "home" page, then it's None
-2. if we're looking at a specific hyperdb class, it's a
-`hyperdb class wrapper`_.
-3. if we're looking at a specific hyperdb item, it's a
-`hyperdb item wrapper`_.
-If the context is not None, we can access the properties of the class or
-item. The only real difference between cases 2 and 3 above are:
-1. the properties may have a real value behind them, and this will
-appear if the property is displayed through ``context/property`` or
-``context/property/field``.
-2. the context's "id" property will be a false value in the second case,
-but a real, or true value in the third. Thus we can determine whether
-we're looking at a real item from the hyperdb by testing
-"context/id".
-Hyperdb class wrapper
-:::::::::::::::::::::
-This is implemented by the ``roundup.cgi.templating.HTMLClass``
-class.
-This wrapper object provides access to a hyperdb class. It is used
-primarily in both index view and new item views, but it's also usable
-anywhere else that you wish to access information about a class, or the
-items of a class, when you don't have a specific item of that class in
-mind.
-We allow access to properties. There will be no "id" property. The value
-accessed through the property will be the current value of the same name
-from the CGI form.
-There are several methods available on these wrapper objects:
-=========== =============================================================
-Method      Description
-=========== =============================================================
-properties  return a `hyperdb property wrapper`_ for all of this class's
-properties that are searchable by the user. You can use
-the argument cansearch=False to get all properties.
-list        lists all of the active (not retired) items in the class.
-csv         return the items of this class as a chunk of CSV text.
-propnames   lists the names of the properties of this class.
-filter      lists of items from this class, filtered and sorted. Two
-options are available for sorting:
-1. by the current *request* filterspec/filter/sort/group args
-2. by the "filterspec", "sort" and "group" keyword args.
-"filterspec" is ``{propname: value(s)}``. "sort" and
-"group" are an optionally empty list ``[(dir, prop)]``
-where dir is '+', '-' or None
-and prop is a prop name or None.
-The propname in filterspec and prop in a sort/group spec
-may be transitive, i.e., it may contain properties of
-the form link.link.link.name.
-eg. All issues with a priority of "1" with messages added in
-the last week, sorted by activity date:
-``issue.filter(filterspec={"priority": "1",
-'messages.creation' : '.-1w;'}, sort=[('activity', '+')])``
-Note that when searching for Link and Multilink values, the
-special value '-1' searches for empty Link or Multilink
-values. For both, Links and Multilinks, multiple values
-given in a filter call are combined with 'OR' by default.
-For Multilinks a postfix expression syntax using negative ID
-numbers (as strings) as operators is supported. Each
-non-negative number (or '-1') is pushed on an operand stack.
-A negative number pops the required number of arguments from
-the stack, applies the operator, and pushes the result. The
-following operators are supported:
-	    - '-2' stands for 'NOT' and takes one argument
-	    - '-3' stands for 'AND' and takes two arguments
-	    - '-4' stands for 'OR' and takes two arguments
-	    Note that this special handling of ID arguments is applied only
-	    when a negative number smaller than -1 is encountered as an ID
-	    in the filter call. Otherwise the implicit OR default
-applies.
-	    Examples of using Multilink expressions would be
-	    - '1', '2', '-4', '3', '4', '-4', '-3'
-	      would search for IDs (1 or 2) and (3 or 4)
-	    - '-1' '-2' would search for all non-empty Multilinks
-filter_sql  **Only in SQL backends**
-Lists the items that match the SQL provided. The SQL is a
-complete "select" statement.
-The SQL select must include the item id as the first column.
-This function **does not** filter out retired items, add
-on a where clause "__retired__ <> 1" if you don't want
-retired nodes.
-classhelp   display a link to a javascript popup containing this class'
-"help" template.
-This generates a link to a popup window which displays the
-properties indicated by "properties" of the class named by
-"classname". The "properties" should be a comma-separated list
-(eg. 'id,name,description'). Properties defaults to all the
-properties of a class (excluding id, creator, created and
-activity).
-You may optionally override the "label" displayed, the "width",
-the "height", the number of items per page ("pagesize") and
-the field on which the list is sorted ("sort").
-With the "filter" arg it is possible to specify a filter for
-which items are supposed to be displayed. It has to be of
-the format "<field>=<values>;<field>=<values>;...".
-The popup window will be resizable and scrollable.
-If the "property" arg is given, it's passed through to the
-javascript help_window function. This allows updating of a
-property in the calling HTML page.
-If the "form" arg is given, it's passed through to the
-javascript help_window function - it's the name of the form
-the "property" belongs to.
-submit      generate a submit button (and action and @csrf hidden elements)
-renderWith  render this class with the given template.
-history     returns 'New node - no history' :)
-is_edit_ok  is the user allowed to Edit the current class?
-is_view_ok  is the user allowed to View the current class?
-=========== =============================================================
-Note that if you have a property of the same name as one of the above
-methods, you'll need to access it using a python "item access"
-expression. For example::
-python:context['list']
-will access the "list" property, rather than the list method.
-Hyperdb item wrapper
-::::::::::::::::::::
-This is implemented by the ``roundup.cgi.templating.HTMLItem``
-class.
-This wrapper object provides access to a hyperdb item.
-We allow access to properties. There will be no "id" property. The value
-accessed through the property will be the current value of the same name
-from the CGI form.
-There are several methods available on these wrapper objects:
-=============== ========================================================
-Method          Description
-=============== ========================================================
-submit          generate a submit button (and action and @csrf hidden elements)
-journal         return the journal of the current item (**not
-implemented**)
-history         render the journal of the current item as
-HTML. By default properties marked as "quiet" (see
-		`design documentation`_) are not shown unless the
-		function is called with the showall=True parameter.
-		Properties that are not Viewable to the user are not
-		shown.
-renderQueryForm specific to the "query" class - render the search form
-for the query
-hasPermission   specific to the "user" class - determine whether the
-user has a Permission. The signature is::
-hasPermission(self, permission, [classname=],
-[property=], [itemid=])
-where the classname defaults to the current context.
-hasRole         specific to the "user" class - determine whether the
-user has a Role. The signature is::
-hasRole(self, rolename)
-is_edit_ok      is the user allowed to Edit the current item?
-is_view_ok      is the user allowed to View the current item?
-is_retired      is the item retired?
-download_url    generate a url-quoted link for download of FileClass
-item contents (ie. file<id>/<name>)
-copy_url        generate a url-quoted link for creating a copy
-of this item.  By default, the copy will acquire
-all properties of the current item except for
-``messages`` and ``files``.  This can be overridden
-by passing ``exclude`` argument which contains a list
-(or any iterable) of property names that shall not be
-copied.  Database-driven properties like ``id`` or
-``activity`` cannot be copied.
-=============== ========================================================
-Note that if you have a property of the same name as one of the above
-methods, you'll need to access it using a python "item access"
-expression. For example::
-python:context['journal']
-will access the "journal" property, rather than the journal method.
-Hyperdb property wrapper
-::::::::::::::::::::::::
-This is implemented by subclasses of the
-``roundup.cgi.templating.HTMLProperty`` class (``HTMLStringProperty``,
-``HTMLNumberProperty``, and so on).
-This wrapper object provides access to a single property of a class. Its
-value may be either:
-1. if accessed through a `hyperdb item wrapper`_, then it's a value from
-the hyperdb
-2. if access through a `hyperdb class wrapper`_, then it's a value from
-the CGI form
-The property wrapper has some useful attributes:
-=============== ========================================================
-Attribute       Description
-=============== ========================================================
-_name           the name of the property
-_value          the value of the property if any - this is the actual
-value retrieved from the hyperdb for this property
-=============== ========================================================
-There are several methods available on these wrapper objects:
-=========== ================================================================
-Method      Description
-=========== ================================================================
-plain       render a "plain" representation of the property. This method
-may take two arguments:
-escape
-If true, escape the text so it is HTML safe (default: no). The
-reason this defaults to off is that text is usually escaped
-at a later stage by the TAL commands, unless the "structure"
-option is used in the template. The following ``tal:content``
-expressions are all equivalent::
-"structure python:msg.content.plain(escape=1)"
-"python:msg.content.plain()"
-"msg/content/plain"
-"msg/content"
-Usually you'll only want to use the escape option in a
-complex expression.
-hyperlink
-If true, turn URLs, email addresses and hyperdb item
-designators in the text into hyperlinks (default: no). Note
-that you'll need to use the "structure" TAL option if you
-want to use this ``tal:content`` expression::
-"structure python:msg.content.plain(hyperlink=1)"
-The text is automatically HTML-escaped before the hyperlinking
-transformation done in the plain() method.
-hyperlinked The same as msg.content.plain(hyperlink=1), but nicer::
-"structure msg/content/hyperlinked"
-field       render an appropriate form edit field for the property - for
-most types this is a text entry box, but for Booleans it's a
-tri-state yes/no/neither selection. This method may take some
-arguments:
-size
-Sets the width in characters of the edit field
-format (Date properties only)
-Sets the format of the date in the field - uses the same
-format string argument as supplied to the ``pretty`` method
-below.
-popcal (Date properties only)
-Include the Javascript-based popup calendar for date
-selection. Defaults to on.
-stext       only on String properties - render the value of the property
-as StructuredText (requires the StructureText module to be
-installed separately)
-multiline   only on String properties - render a multiline form edit
-field for the property
-email       only on String properties - render the value of the property
-as an obscured email address
-url_quote   only on String properties. It quotes any characters in the
-string so it is safe to use in a url. E.G. a space is
-replaced with %20.
-confirm     only on Password properties - render a second form edit field
-for the property, used for confirmation that the user typed
-the password correctly. Generates a field with name
-"name:confirm".
-now         only on Date properties - return the current date as a new
-property
-reldate     only on Date properties - render the interval between the date
-and now
-local       only on Date properties - return this date as a new property
-with some timezone offset, for example::
-python:context.creation.local(10)
-will render the date with a +10 hour offset.
-pretty      Date properties - render the date as "dd Mon YYYY" (eg. "19
-Mar 2004"). Takes an optional format argument, for example::
-python:context.activity.pretty('%Y-%m-%d')
-Will format as "2004-03-19" instead.
-Interval properties - render the interval in a pretty
-format (eg. "yesterday"). The format arguments are those used
-in the standard ``strftime`` call (see the `Python Library
-Reference: time module`__)
-Number properties - takes a printf style format argument
-(default: '%0.3f') and formats the number accordingly.
-If the value can't be converted, '' is returned if the
-value is ``None`` otherwise it is converted to a string.
-popcal      Generate a link to a popup calendar which may be used to
-edit the date field, for example::
-<span tal:replace="structure context/due/popcal" />
-you still need to include the ``field`` for the property, so
-typically you'd have::
-<span tal:replace="structure context/due/field" />
-<span tal:replace="structure context/due/popcal" />
-menu        only on Link and Multilink properties - render a form select
-list for this property. Takes a number of optional arguments
-size
-is used to limit the length of the list labels
-height
-is used to set the <select> tag's "size" attribute
-showid
-includes the item ids in the list labels
-additional
-lists properties which should be included in the label
-sort_on
-indicates the property to sort the list on as (direction,
-(direction, property) where direction is '+' or '-'. A
-single string with the direction prepended may be used.
-For example: ('-', 'order'), '+name'.
-value
-gives a default value to preselect in the menu
-The remaining keyword arguments are used as conditions for
-filtering the items in the list - they're passed as the
-"filterspec" argument to a Class.filter() call. For example::
-<span tal:replace="structure context/status/menu" />
-<span tal:replace="python:context.status.menu(order='+name",
-value='chatting',
-filterspec={'status': '1,2,3,4'}" />
-sorted      only on Multilink properties - produce a list of the linked
-items sorted by some property, for example::
-python:context.files.sorted('creation')
-Will list the files by upload date. While
-python:context.files.sorted('creation', reverse=True)
-Will list the files by upload date in reverse order from
-the prior example. If the property can be unset, you can
-use the ``NoneFirst`` parameter to sort the None/Unset
-values at the front or the end of the list. For example::
-python:context.files.sorted('creation', NoneFirst=True)
-will sort files by creation date with files missing a
-creation date at the start of the list. The default for
-``NoneFirst`` is False so these files will sort at the end
-by default. (Note creation date is never unset, but you
-get the idea.) If you combine NoneFirst with
-``reverse=True`` the meaning of NoneFirst is inverted:
-	    True sorts None/unset at the end and False sorts at the
-	    beginning.
-reverse     only on Multilink properties - produce a list of the linked
-items in reverse order
-isset       returns True if the property has been set to a value
-=========== ================================================================
-__ https://docs.python.org/2/library/time.html
-All of the above functions perform checks for permissions required to
-display or edit the data they are manipulating. The simplest case is
-editing an issue title. Including the expression::
-context/title/field
-Will present the user with an edit field, if they have edit permission. If
-not, then they will be presented with a static display if they have view
-permission. If they don't even have view permission, then an error message
-is raised, preventing the display of the page, indicating that they don't
-have permission to view the information.
-The request variable
-~~~~~~~~~~~~~~~~~~~~
-This is implemented by the ``roundup.cgi.templating.HTMLRequest``
-class.
-The request variable is packed with information about the current
-request.
-.. taken from ``roundup.cgi.templating.HTMLRequest`` docstring
-=========== ============================================================
-Variable    Holds
-=========== ============================================================
-form        the CGI form as a cgi.FieldStorage
-env         the CGI environment variables
-base        the base URL for this tracker
-user        a HTMLUser instance for this user
-classname   the current classname (possibly None)
-template    the current template (suffix, also possibly None)
-form        the current CGI form variables in a FieldStorage
-=========== ============================================================
-**Index page specific variables (indexing arguments)**
-=========== ============================================================
-Variable    Holds
-=========== ============================================================
-columns     dictionary of the columns to display in an index page
-show        a convenience access to columns - request/show/colname will
-be true if the columns should be displayed, false otherwise
-sort        index sort columns [(direction, column name)]
-group       index grouping properties [(direction, column name)]
-filter      properties to filter the index on
-filterspec  values to filter the index on (property=value, eg
-``priority=1`` or ``messages.author=42``
-search_text text to perform a full-text search on for an index
-=========== ============================================================
-There are several methods available on the request variable:
-=============== ========================================================
-Method          Description
-=============== ========================================================
-description     render a description of the request - handle for the
-page title
-indexargs_form  render the current index args as form elements
-indexargs_url   render the current index args as a URL
-base_javascript render some javascript that is used by other components
-of the templating
-batch           run the current index args through a filter and return a
-list of items (see `hyperdb item wrapper`_, and
-`batching`_)
-=============== ========================================================
-The form variable
-:::::::::::::::::
-The form variable is a bit special because it's actually a python
-FieldStorage object. That means that you have two ways to access its
-contents. For example, to look up the CGI form value for the variable
-"name", use the path expression::
-request/form/name/value
-or the python expression::
-python:request.form['name'].value
-Note the "item" access used in the python case, and also note the
-explicit "value" attribute we have to access. That's because the form
-variables are stored as MiniFieldStorages. If there's more than one
-"name" value in the form, then the above will break since
-``request/form/name`` is actually a *list* of MiniFieldStorages. So it's
-best to know beforehand what you're dealing with.
-The db variable
-~~~~~~~~~~~~~~~
-This is implemented by the ``roundup.cgi.templating.HTMLDatabase``
-class.
-Allows access to all hyperdb classes as attributes of this variable. If
-you want access to the "user" class, for example, you would use::
-db/user
-python:db.user
-Also, the current id of the current user is available as
-``db.getuid()``. This isn't so useful in templates (where you have
-``request/user``), but it can be useful in detectors or interfaces.
-The access results in a `hyperdb class wrapper`_.
-The templates variable
-~~~~~~~~~~~~~~~~~~~~~~
-This was implemented by the ``roundup.cgi.templating.Templates``
-class before 1.4.20. In later versions it is the instance of appropriate
-template engine loader class.
-This variable is used to access other templates in expressions and
-template macros. It doesn't have any useful methods defined. The
-templates can be accessed using the following path expression::
-templates/name
-or the python expression::
-templates[name]
-where "name" is the name of the template you wish to access. The
-template has one useful attribute, namely "macros". To access a specific
-macro (called "macro_name"), use the path expression::
-templates/name/macros/macro_name
-or the python expression::
-templates[name].macros[macro_name]
-The repeat variable
-~~~~~~~~~~~~~~~~~~~
-The repeat variable holds an entry for each active iteration. That is, if
-you have a ``tal:repeat="user db/users"`` command, then there will be a
-repeat variable entry called "user". This may be accessed as either::
-repeat/user
-python:repeat['user']
-The "user" entry has a number of methods available for information:
-=============== =========================================================
-Method          Description
-=============== =========================================================
-first           True if the current item is the first in the sequence.
-last            True if the current item is the last in the sequence.
-even            True if the current item is an even item in the sequence.
-odd             True if the current item is an odd item in the sequence.
-number          Current position in the sequence, starting from 1.
-letter          Current position in the sequence as a letter, a through
-z, then aa through zz, and so on.
-Letter          Same as letter(), except uppercase.
-roman           Current position in the sequence as lowercase roman
-numerals.
-Roman           Same as roman(), except uppercase.
-=============== =========================================================
-The utils variable
-~~~~~~~~~~~~~~~~~~
-.. _templating utilities:
-This is implemented by the
-``roundup.cgi.templating.TemplatingUtils`` class, which may be extended
-with additional methods by extensions_.
-=============== ========================================================
-Method          Description
-=============== ========================================================
-Batch           return a batch object using the supplied list
-url_quote       quote some text as safe for a URL (ie. space, %, ...)
-html_quote      quote some text as safe in HTML (ie. <, >, ...)
-html_calendar   renders an HTML calendar used by the
-``_generic.calendar.html`` template (itself invoked by
-the popupCalendar DateHTMLProperty method
-anti_csrf_nonce returns the random noncue generated for this session
-=============== ========================================================
-Batching
-::::::::
-Use Batch to turn a list of items, or item ids of a given class, into a
-series of batches. Its usage is::
-python:utils.Batch(sequence, size, start, end=0, orphan=0,
-overlap=0)
-or, to get the current index batch::
-request/batch
-The parameters are:
-========= ==============================================================
-Parameter  Usage
-========= ==============================================================
-sequence  a list of HTMLItems
-size      how big to make the sequence.
-start     where to start (0-indexed) in the sequence.
-end       where to end (0-indexed) in the sequence.
-orphan    if the next batch would contain less items than this value,
-then it is combined with this batch
-overlap   the number of items shared between adjacent batches
-========= ==============================================================
-All of the parameters are assigned as attributes on the batch object. In
-addition, it has several more attributes:
-=============== ========================================================
-Attribute       Description
-=============== ========================================================
-start           indicates the start index of the batch. *Unlike
-the argument, is a 1-based index (I know, lame)*
-first           indicates the start index of the batch *as a 0-based
-index*
-length          the actual number of elements in the batch
-sequence_length the length of the original, unbatched, sequence.
-=============== ========================================================
-And several methods:
-=============== ========================================================
-Method          Description
-=============== ========================================================
-previous        returns a new Batch with the previous batch settings
-next            returns a new Batch with the next batch settings
-propchanged     detect if the named property changed on the current item
-when compared to the last item
-=============== ========================================================
-An example of batching::
-<table class="otherinfo">
-<tr><th colspan="4" class="header">Existing Keywords</th></tr>
-<tr tal:define="keywords db/keyword/list"
-tal:repeat="start python:range(0, len(keywords), 4)">
-<td tal:define="batch python:utils.Batch(keywords, 4, start)"
-tal:repeat="keyword batch" tal:content="keyword/name">
-keyword here</td>
-</tr>
-</table>
-... which will produce a table with four columns containing the items of
-the "keyword" class (well, their "name" anyway).
-Translations
-~~~~~~~~~~~~
-Should you wish to enable multiple languages in template content that you
-create you'll need to add new locale files in the tracker home under a
-``locale`` directory. Use the instructions in the ``developer's guide`` to
-create the locale files.
-Displaying Properties
----------------------
-Properties appear in the user interface in three contexts: in indices,
-in editors, and as search arguments. For each type of property, there
-are several display possibilities. For example, in an index view, a
-string property may just be printed as a plain string, but in an editor
-view, that property may be displayed in an editable field.
-Index Views
------------
-This is one of the class context views. It is also the default view for
-classes. The template used is "*classname*.index".
-Index View Specifiers
-~~~~~~~~~~~~~~~~~~~~~
-An index view specifier (URL fragment) looks like this (whitespace has
-been added for clarity)::
-/issue?status=unread,in-progress,resolved&
-keyword=security,ui&
-@group=priority,-status&
-@sort=-activity&
-@filters=status,keyword&
-@columns=title,status,fixer
-The index view is determined by two parts of the specifier: the layout
-part and the filter part. The layout part consists of the query
-parameters that begin with colons, and it determines the way that the
-properties of selected items are displayed. The filter part consists of
-all the other query parameters, and it determines the criteria by which
-items are selected for display. The filter part is interactively
-manipulated with the form widgets displayed in the filter section. The
-layout part is interactively manipulated by clicking on the column
-headings in the table.
-The filter part selects the union of the sets of items with values
-matching any specified Link properties and the intersection of the sets
-of items with values matching any specified Multilink properties.
-The example specifies an index of "issue" items. Only items with a
-"status" of either "unread" or "in-progress" or "resolved" are
-displayed, and only items with "keyword" values including both "security"
-and "ui" are displayed. The items are grouped by priority arranged in
-ascending order and in descending order by status; and within
-groups, sorted by activity, arranged in descending order. The filter
-section shows filters for the "status" and "keyword" properties, and the
-table includes columns for the "title", "status", and "fixer"
-properties.
-============ =============================================================
-Argument     Description
-============ =============================================================
-@sort        sort by prop name, optionally preceeded with '-' to give
-descending or nothing for ascending sorting. Several
-properties can be specified delimited with comma.
-Internally a search-page using several sort properties may
-use @sort0, @sort1 etc. with option @sortdir0, @sortdir1
-etc. for the direction of sorting (a non-empty value of
-sortdir0 specifies reverse order).
-@group       group by prop name, optionally preceeded with '-' or to sort
-in descending or nothing for ascending order. Several
-properties can be specified delimited with comma.
-Internally a search-page using several grouping properties may
-use @group0, @group1 etc. with option @groupdir0, @groupdir1
-etc. for the direction of grouping (a non-empty value of
-groupdir0 specifies reverse order).
-@columns     selects the columns that should be displayed. Default is
-all.
-@filter      indicates which properties are being used in filtering.
-Default is none.
-propname     selects the values the item properties given by propname must
-have (very basic search/filter).
-@search_text if supplied, performs a full-text search (message bodies,
-issue titles, etc)
-============ =============================================================
-Searching Views
----------------
-.. note::
-if you add a new column to the ``@columns`` form variable potentials
-then you will need to add the column to the appropriate `index views`_
-template so that it is actually displayed.
-This is one of the class context views. The template used is typically
-"*classname*.search". The form on this page should have "search" as its
-``@action`` variable. The "search" action:
-- sets up additional filtering, as well as performing indexed text
-searching
-- sets the ``@filter`` variable correctly
-- saves the query off if ``@query_name`` is set.
-The search page should lay out any fields that you wish to allow the
-user to search on. If your schema contains a large number of properties,
-you should be wary of making all of those properties available for
-searching, as this can cause confusion. If the additional properties are
-Strings, consider having their value indexed, and then they will be
-searchable using the full text indexed search. This is both faster, and
-more useful for the end user.
-If the search view does specify the "search" ``@action``, then it may also
-provide an additional argument:
-============ =============================================================
-Argument     Description
-============ =============================================================
-@query_name  if supplied, the index parameters (including @search_text)
-will be saved off as a the query item and registered against
-the user's queries property. Note that the *classic* template
-schema has this ability, but the *minimal* template schema
-does not.
-============ =============================================================
-Item Views
-----------
-The basic view of a hyperdb item is provided by the "*classname*.item"
-template. It generally has three sections; an "editor", a "spool" and a
-"history" section.
-Editor Section
-~~~~~~~~~~~~~~
-The editor section is used to manipulate the item - it may be a static
-display if the user doesn't have permission to edit the item.
-Here's an example of a basic editor template (this is the default
-"classic" template issue item edit form - from the "issue.item.html"
-template)::
-<table class="form">
-<tr>
-<th>Title</th>
-<td colspan="3" tal:content="structure python:context.title.field(size=60)">title</td>
-</tr>
-<tr>
-<th>Priority</th>
-<td tal:content="structure context/priority/menu">priority</td>
-<th>Status</th>
-<td tal:content="structure context/status/menu">status</td>
-</tr>
-<tr>
-<th>Superseder</th>
-<td>
-<span tal:replace="structure python:context.superseder.field(showid=1, size=20)" />
-<span tal:replace="structure python:db.issue.classhelp('id,title')" />
-<span tal:condition="context/superseder">
-<br>View: <span tal:replace="structure python:context.superseder.link(showid=1)" />
-</span>
-</td>
-<th>Nosy List</th>
-<td>
-<span tal:replace="structure context/nosy/field" />
-<span tal:replace="structure python:db.user.classhelp('username,realname,address,phone')" />
-</td>
-</tr>
-<tr>
-<th>Assigned To</th>
-<td tal:content="structure context/assignedto/menu">
-assignedto menu
-</td>
-<td>&nbsp;</td>
-<td>&nbsp;</td>
-</tr>
-<tr>
-<th>Change Note</th>
-<td colspan="3">
-<textarea name=":note" wrap="hard" rows="5" cols="60"></textarea>
-</td>
-</tr>
-<tr>
-<th>File</th>
-<td colspan="3"><input type="file" name=":file" size="40"></td>
-</tr>
-<tr>
-<td>&nbsp;</td>
-<td colspan="3" tal:content="structure context/submit">
-submit button will go here
-</td>
-</tr>
-</table>
-When a change is submitted, the system automatically generates a message
-describing the changed properties. As shown in the example, the editor
-template can use the ":note" and ":file" fields, which are added to the
-standard changenote message generated by Roundup.
-Form values
-:::::::::::
-We have a number of ways to pull properties out of the form in order to
-meet the various needs of:
-1. editing the current item (perhaps an issue item)
-2. editing information related to the current item (eg. messages or
-attached files)
-3. creating new information to be linked to the current item (eg. time
-spent on an issue)
-In the following, ``<bracketed>`` values are variable, ":" may be one of
-":" or "@", and other text ("required") is fixed.
-Properties are specified as form variables:
-``<propname>``
-property on the current context item
-``<designator>:<propname>``
-property on the indicated item (for editing related information)
-``<classname>-<N>:<propname>``
-property on the Nth new item of classname (generally for creating new
-items to attach to the current item)
-Once we have determined the "propname", we check to see if it is one of
-the special form values:
-``@required``
-The named property values must be supplied or a ValueError will be
-raised.
-``@remove@<propname>=id(s)``
-The ids will be removed from the multilink property.
-``:add:<propname>=id(s)``
-The ids will be added to the multilink property.
-``:link:<propname>=<designator>``
-Used to add a link to new items created during edit. These are
-collected and returned in ``all_links``. This will result in an
-additional linking operation (either Link set or Multilink append)
-after the edit/create is done using ``all_props`` in ``_editnodes``.
-The <propname> on the current item will be set/appended the id of the
-newly created item of class <designator> (where <designator> must be
-<classname>-<N>).
-Any of the form variables may be prefixed with a classname or
-designator.
-Two special form values are supported for backwards compatibility:
-``:note``
-create a message (with content, author and date), linked to the
-context item. This is ALWAYS designated "msg-1".
-``:file``
-create a file, attached to the current item and any message created by
-:note. This is ALWAYS designated "file-1".
-Spool Section
-~~~~~~~~~~~~~
-The spool section lists related information like the messages and files
-of an issue.
-TODO
-History Section
-~~~~~~~~~~~~~~~
-The final section displayed is the history of the item - its database
-journal. This is generally generated with the template::
-<tal:block tal:replace="structure context/history" />
-or::
-<tal:block
-tal:replace="structure python:context.history(showall=True)" />
-if you want to show history entries for quiet properties.
-*To be done:*
-*The actual history entries of the item may be accessed for manual
-templating through the "journal" method of the item*::
-<tal:block tal:repeat="entry context/journal">
-a journal entry
-</tal:block>
-*where each journal entry is an HTMLJournalEntry.*
-Defining new web actions
-------------------------
-You may define new actions to be triggered by the ``@action`` form variable.
-These are added to the tracker ``extensions`` directory and registered
-using ``instance.registerAction``.
-All the existing Actions are defined in ``roundup.cgi.actions``.
-Adding action classes takes three steps; first you `define the new
-action class`_, then you `register the action class`_ with the cgi
-interface so it may be triggered by the ``@action`` form variable.
-Finally you `use the new action`_ in your HTML form.
-See "`setting up a "wizard" (or "druid") for controlled adding of
-issues`_" for an example.
-Define the new action class
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Create a new action class in your tracker's ``extensions`` directory, for
-example ``myaction.py``::
-from roundup.cgi.actions import Action
-class MyAction(Action):
-def handle(self):
-''' Perform some action. No return value is required.
-'''
-The *self.client* attribute is an instance of ``roundup.cgi.client.Client``.
-See the docstring of that class for details of what it can do.
-The method will typically check the ``self.form`` variable's contents.
-It may then:
-- add information to ``self.client._ok_message``
-or ``self.client._error_message`` (by using ``self.client.add_ok_message``
-or ``self.client.add_error_message``, respectively)
-- change the ``self.client.template`` variable to alter what the user will see
-next
-- raise Unauthorised, SendStaticFile, SendFile, NotFound or Redirect
-exceptions (import them from roundup.cgi.exceptions)
-Register the action class
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-The class is now written, but isn't available to the user until you register
-it with the following code appended to your ``myaction.py`` file::
-def init(instance):
-instance.registerAction('myaction', myActionClass)
-This maps the action name "myaction" to the action class we defined.
-Use the new action
-~~~~~~~~~~~~~~~~~~
-In your HTML form, add a hidden form element like so::
-<input type="hidden" name="@action" value="myaction">
-where "myaction" is the name you registered in the previous step.
-Actions may return content to the user
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Actions generally perform some database manipulation and then pass control
-on to the rendering of a template in the current context (see `Determining
-web context`_ for how that works.) Some actions will want to generate the
-actual content returned to the user. Action methods may return their own
-content string to be displayed to the user, overriding the templating step.
-In this situation, we assume that the content is HTML by default. You may
-override the content type indicated to the user by calling ``setHeader``::
-self.client.setHeader('Content-Type', 'text/csv')
-This example indicates that the value sent back to the user is actually
-comma-separated value content (eg. something to be loaded into a
-spreadsheet or database).
-8-bit character set support in Web interface
---------------------------------------------
-The web interface uses UTF-8 default. It may be overridden in both forms
-and a browser cookie.
-- In forms, use the ``@charset`` variable.
-- To use the cookie override, have the ``roundup_charset`` cookie set.
-In both cases, the value is a valid charset name (eg. ``utf-8`` or
-``kio8-r``).
-Inside Roundup, all strings are stored and processed in utf-8.
-Unfortunately, some older browsers do not work properly with
-utf-8-encoded pages (e.g. Netscape Navigator 4 displays wrong
-characters in form fields).  This version allows one to change
-the character set for http transfers.  To do so, you may add
-the following code to your ``page.html`` template::
-<tal:block define="uri string:${request/base}${request/env/PATH_INFO}">
-<a tal:attributes="href python:request.indexargs_url(uri,
-{'@charset':'utf-8'})">utf-8</a>
-<a tal:attributes="href python:request.indexargs_url(uri,
-{'@charset':'koi8-r'})">koi8-r</a>
-</tal:block>
-(substitute ``koi8-r`` with appropriate charset for your language).
-Charset preference is kept in the browser cookie ``roundup_charset``.
-``meta http-equiv`` lines added to the tracker templates in version 0.6.0
-should be changed to include actual character set name::
-<meta http-equiv="Content-Type"
-tal:attributes="content string:text/html;; charset=${request/client/charset}"
-/>
-The charset is also sent in the http header.
-.. _CustomExamples:
-Examples
-========
-.. contents::
-:local:
-:depth: 2
-Changing what's stored in the database
---------------------------------------
-The following examples illustrate ways to change the information stored in
-the database.
-Adding a new field to the classic schema
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-This example shows how to add a simple field (a due date) to the default
-classic schema. It does not add any additional behaviour, such as enforcing
-the due date, or causing automatic actions to fire if the due date passes.
-You add new fields by editing the ``schema.py`` file in you tracker's home.
-Schema changes are automatically applied to the database on the next
-tracker access (note that roundup-server would need to be restarted as it
-caches the schema).
-.. index:: schema; example changes
-1. Modify the ``schema.py``::
-issue = IssueClass(db, "issue",
-assignedto=Link("user"), keyword=Multilink("keyword"),
-priority=Link("priority"), status=Link("status"),
-due_date=Date())
-2. Add an edit field to the ``issue.item.html`` template::
-<tr>
-<th>Due Date</th>
-<td tal:content="structure context/due_date/field" />
-</tr>
-If you want to show only the date part of due_date then do this instead::
-<tr>
-<th>Due Date</th>
-<td tal:content="structure python:context.due_date.field(format='%Y-%m-%d')" />
-</tr>
-3. Add the property to the ``issue.index.html`` page::
-(in the heading row)
-<th tal:condition="request/show/due_date">Due Date</th>
-(in the data row)
-<td tal:condition="request/show/due_date"
-tal:content="i/due_date" />
-If you want format control of the display of the due date you can
-enter the following in the data row to show only the actual due date::
-<td tal:condition="request/show/due_date"
-tal:content="python:i.due_date.pretty('%Y-%m-%d')">&nbsp;</td>
-4. Add the property to the ``issue.search.html`` page::
-<tr tal:define="name string:due_date">
-<th i18n:translate="">Due Date:</th>
-<td metal:use-macro="search_input"></td>
-<td metal:use-macro="column_input"></td>
-<td metal:use-macro="sort_input"></td>
-<td metal:use-macro="group_input"></td>
-</tr>
-5. If you wish for the due date to appear in the standard views listed
-in the sidebar of the web interface then you'll need to add "due_date"
-to the columns and columns_showall lists in your ``page.html``::
-columns string:id,activity,due_date,title,creator,status;
-columns_showall string:id,activity,due_date,title,creator,assignedto,status;
-Adding a new Computed field to the schema
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Computed properties are a bit different from other properties. They do
-not actually change the database. Computed fields are not contained in
-the database and can not be searched or used for sorting or
-grouping. (Patches to add this capability are welcome.)
-In this example we will add a count of the number of files attached to
-the issue. This could be done using an auditor (see below) to update
-an integer field called ``filecount``. But we will implement this in a
-different way.
-We have two changes to make:
-1. add a new python method to the hyperdb.Computed class. It will
-count the number of files attached to the issue. This method will
-be added in the (possibly new) interfaces.py file in the top level
-of the tracker directory.  (See interfaces.py above for more
-information.)
-2. add a new ``filecount`` property to the issue class calling the
-new function.
-A Computed method receives three arguments when called:
-1. the computed object (self)
-2. the id of the item in the class
-3. the database object
-To add the method to the Computed class, modify the trackers
-interfaces.py adding::
-import roundup.hyperdb as hyperdb
-def filecount(self, nodeid, db):
-return len(db.issue.get(nodeid, 'files'))
-setattr(hyperdb.Computed, 'filecount', filecount)
-Then add::
-filecount = Computed(Computed.filecount),
-to the existing IssueClass call.
-Now you can retrieve the value of the ``filecount`` property and it
-will be computed on the fly from the existing list of attached files.
-This example was done with the IssueClass, but you could add a
-Computed property to any class. E.G.::
-user = Class(db, "user",
-username=String(),
-password=Password(),
-address=String(),
-realname=String(),
-phone=String(),
-office=Computed(Computed.getOfficeFromLdap), # new prop
-organisation=String(),
-alternate_addresses=String(),
-queries=Multilink('query'),
-roles=String(),
-timezone=String())
-where the method ``getOfficeFromLdap`` queries the local ldap server to
-get the current office location information. The method will be called
-with the Computed instance, the user id and the database object.
-Adding a new constrained field to the classic schema
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-This example shows how to add a new constrained property (i.e. a
-selection of distinct values) to your tracker.
-Introduction
-::::::::::::
-To make the classic schema of Roundup useful as a TODO tracking system
-for a group of systems administrators, it needs an extra data field per
-issue: a category.
-This would let sysadmins quickly list all TODOs in their particular area
-of interest without having to do complex queries, and without relying on
-the spelling capabilities of other sysadmins (a losing proposition at
-best).
-Adding a field to the database
-::::::::::::::::::::::::::::::
-This is the easiest part of the change. The category would just be a
-plain string, nothing fancy. To change what is in the database you need
-to add some lines to the ``schema.py`` file of your tracker instance.
-Under the comment::
-# add any additional database schema configuration here
-add::
-category = Class(db, "category", name=String())
-category.setkey("name")
-Here we are setting up a chunk of the database which we are calling
-"category". It contains a string, which we are refering to as "name" for
-lack of a more imaginative title. (Since "name" is one of the properties
-that Roundup looks for on items if you do not set a key for them, it's
-probably a good idea to stick with it for new classes if at all
-appropriate.) Then we are setting the key of this chunk of the database
-to be that "name". This is equivalent to an index for database types.
-This also means that there can only be one category with a given name.
-Adding the above lines allows us to create categories, but they're not
-tied to the issues that we are going to be creating. It's just a list of
-categories off on its own, which isn't much use. We need to link it in
-with the issues. To do that, find the lines
-in ``schema.py`` which set up the "issue" class, and then add a link to
-the category::
-issue = IssueClass(db, "issue", ... ,
-category=Multilink("category"), ... )
-The ``Multilink()`` means that each issue can have many categories. If
-you were adding something with a one-to-one relationship to issues (such
-as the "assignedto" property), use ``Link()`` instead.
-That is all you need to do to change the schema. The rest of the effort
-is fiddling around so you can actually use the new category.
-Populating the new category class
-:::::::::::::::::::::::::::::::::
-If you haven't initialised the database with the
-"``roundup-admin initialise``" command, then you
-can add the following to the tracker ``initial_data.py``
-under the comment::
-# add any additional database creation steps here - but only if you
-# haven't initialised the database with the admin "initialise" command
-Add::
-category = db.getclass('category')
-category.create(name="scipy")
-category.create(name="chaco")
-category.create(name="weave")
-.. index:: roundup-admin; create entries in class
-If the database has already been initalised, then you need to use the
-``roundup-admin`` tool::
-% roundup-admin -i <tracker home>
-Roundup <version> ready for input.
-Type "help" for help.
-roundup> create category name=scipy
-1
-roundup> create category name=chaco
-2
-roundup> create category name=weave
-3
-roundup> exit...
-There are unsaved changes. Commit them (y/N)? y
-Setting up security on the new objects
-::::::::::::::::::::::::::::::::::::::
-By default only the admin user can look at and change objects. This
-doesn't suit us, as we want any user to be able to create new categories
-as required, and obviously everyone needs to be able to view the
-categories of issues for it to be useful.
-We therefore need to change the security of the category objects. This
-is also done in ``schema.py``.
-There are currently two loops which set up permissions and then assign
-them to various roles. Simply add the new "category" to both lists::
-# Assign the access and edit permissions for issue, file and message
-# to regular users now
-for cl in 'issue', 'file', 'msg', 'category':
-p = db.security.getPermission('View', cl)
-db.security.addPermissionToRole('User', 'View', cl)
-db.security.addPermissionToRole('User', 'Edit', cl)
-db.security.addPermissionToRole('User', 'Create', cl)
-These lines assign the "View" and "Edit" Permissions to the "User" role,
-so that normal users can view and edit "category" objects.
-This is all the work that needs to be done for the database. It will
-store categories, and let users view and edit them. Now on to the
-interface stuff.
-Changing the web left hand frame
-::::::::::::::::::::::::::::::::
-We need to give the users the ability to create new categories, and the
-place to put the link to this functionality is in the left hand function
-bar, under the "Issues" area. The file that defines how this area looks
-is ``html/page.html``, which is what we are going to be editing next.
-If you look at this file you can see that it contains a lot of
-"classblock" sections which are chunks of HTML that will be included or
-excluded in the output depending on whether the condition in the
-classblock is met. We are going to add the category code at the end of
-the classblock for the *issue* class::
-<p class="classblock"
-tal:condition="python:request.user.hasPermission('View', 'category')">
-<b>Categories</b><br>
-<a tal:condition="python:request.user.hasPermission('Edit', 'category')"
-href="category?@template=item">New Category<br></a>
-</p>
-The first two lines is the classblock definition, which sets up a
-condition that only users who have "View" permission for the "category"
-object will have this section included in their output. Next comes a
-plain "Categories" header in bold. Everyone who can view categories will
-get that.
-Next comes the link to the editing area of categories. This link will
-only appear if the condition - that the user has "Edit" permissions for
-the "category" objects - is matched. If they do have permission then
-they will get a link to another page which will let the user add new
-categories.
-Note that if you have permission to *view* but not to *edit* categories,
-then all you will see is a "Categories" header with nothing underneath
-it. This is obviously not very good interface design, but will do for
-now. I just claim that it is so I can add more links in this section
-later on. However, to fix the problem you could change the condition in
-the classblock statement, so that only users with "Edit" permission
-would see the "Categories" stuff.
-Setting up a page to edit categories
-::::::::::::::::::::::::::::::::::::
-We defined code in the previous section which let users with the
-appropriate permissions see a link to a page which would let them edit
-conditions. Now we have to write that page.
-The link was for the *item* template of the *category* object. This
-translates into Roundup looking for a file called ``category.item.html``
-in the ``html`` tracker directory. This is the file that we are going to
-write now.
-First, we add an info tag in a comment which doesn't affect the outcome
-of the code at all, but is useful for debugging. If you load a page in a
-browser and look at the page source, you can see which sections come
-from which files by looking for these comments::
-<!-- category.item -->
-Next we need to add in the METAL macro stuff so we get the normal page
-trappings::
-<tal:block metal:use-macro="templates/page/macros/icing">
-<title metal:fill-slot="head_title">Category editing</title>
-<td class="page-header-top" metal:fill-slot="body_title">
-<h2>Category editing</h2>
-</td>
-<td class="content" metal:fill-slot="content">
-Next we need to setup up a standard HTML form, which is the whole
-purpose of this file. We link to some handy javascript which sends the
-form through only once. This is to stop users hitting the send button
-multiple times when they are impatient and thus having the form sent
-multiple times::
-<form method="POST" onSubmit="return submit_once()"
-enctype="multipart/form-data">
-Next we define some code which sets up the minimum list of fields that
-we require the user to enter. There will be only one field - "name" - so
-they better put something in it, otherwise the whole form is pointless::
-<input type="hidden" name="@required" value="name">
-To get everything to line up properly we will put everything in a table,
-and put a nice big header on it so the user has an idea what is
-happening::
-<table class="form">
-<tr><th class="header" colspan="2">Category</th></tr>
-Next, we need the field into which the user is going to enter the new
-category. The ``context.name.field(size=60)`` bit tells Roundup to
-generate a normal HTML field of size 60, and the contents of that field
-will be the "name" variable of the current context (namely "category").
-The upshot of this is that when the user types something in
-to the form, a new category will be created with that name::
-<tr>
-<th>Name</th>
-<td tal:content="structure python:context.name.field(size=60)">
-name</td>
-</tr>
-Then a submit button so that the user can submit the new category::
-<tr>
-<td>&nbsp;</td>
-<td colspan="3" tal:content="structure context/submit">
-submit button will go here
-</td>
-</tr>
-The ``context/submit`` bit generates the submit button but also
-generates the @action and @csrf hidden fields. The @action field is
-used to tell roundup how to process the form. The @csrf field provides
-a unique single use token to defend against CSRF attacks. (More about
-anti-csrf measures can be found in ``upgrading.txt``.)
-Finally we finish off the tags we used at the start to do the METAL
-stuff::
-</td>
-</tal:block>
-So putting it all together, and closing the table and form we get::
-<!-- category.item -->
-<tal:block metal:use-macro="templates/page/macros/icing">
-<title metal:fill-slot="head_title">Category editing</title>
-<td class="page-header-top" metal:fill-slot="body_title">
-<h2>Category editing</h2>
-</td>
-<td class="content" metal:fill-slot="content">
-<form method="POST" onSubmit="return submit_once()"
-enctype="multipart/form-data">
-<table class="form">
-<tr><th class="header" colspan="2">Category</th></tr>
-<tr>
-<th>Name</th>
-<td tal:content="structure python:context.name.field(size=60)">
-name</td>
-</tr>
-<tr>
-<td>
-&nbsp;
-<input type="hidden" name="@required" value="name">
-</td>
-<td colspan="3" tal:content="structure context/submit">
-submit button will go here
-</td>
-</tr>
-</table>
-</form>
-</td>
-</tal:block>
-This is quite a lot to just ask the user one simple question, but there
-is a lot of setup for basically one line (the form line) to do its work.
-To add another field to "category" would involve one more line (well,
-maybe a few extra to get the formatting correct).
-Adding the category to the issue
-::::::::::::::::::::::::::::::::
-We now have the ability to create issues to our heart's content, but
-that is pointless unless we can assign categories to issues.  Just like
-the ``html/category.item.html`` file was used to define how to add a new
-category, the ``html/issue.item.html`` is used to define how a new issue
-is created.
-Just like ``category.issue.html``, this file defines a form which has a
-table to lay things out. It doesn't matter where in the table we add new
-stuff, it is entirely up to your sense of aesthetics::
-<th>Category</th>
-<td>
-<span tal:replace="structure context/category/field" />
-<span tal:replace="structure python:db.category.classhelp('name',
-property='category', width='200')" />
-</td>
-First, we define a nice header so that the user knows what the next
-section is, then the middle line does what we are most interested in.
-This ``context/category/field`` gets replaced by a field which contains
-the category in the current context (the current context being the new
-issue).
-The classhelp lines generate a link (labelled "list") to a popup window
-which contains the list of currently known categories.
-Searching on categories
-:::::::::::::::::::::::
-Now we can add categories, and create issues with categories. The next
-obvious thing that we would like to be able to do, would be to search
-for issues based on their category, so that, for example, anyone working
-on the web server could look at all issues in the category "Web".
-If you look for "Search Issues" in the ``html/page.html`` file, you will
-find that it looks something like
-``<a href="issue?@template=search">Search Issues</a>``. This shows us
-that when you click on "Search Issues" it will be looking for a
-``issue.search.html`` file to display. So that is the file that we will
-change.
-If you look at this file it should begin to seem familiar, although it
-does use some new macros. You can add the new category search code anywhere you
-like within that form::
-<tr tal:define="name string:category;
-db_klass string:category;
-db_content string:name;">
-<th>Priority:</th>
-<td metal:use-macro="search_select"></td>
-<td metal:use-macro="column_input"></td>
-<td metal:use-macro="sort_input"></td>
-<td metal:use-macro="group_input"></td>
-</tr>
-The definitions in the ``<tr>`` opening tag are used by the macros:
-- ``search_select`` expands to a drop-down box with all categories using
-``db_klass`` and ``db_content``.
-- ``column_input`` expands to a checkbox for selecting what columns
-should be displayed.
-- ``sort_input`` expands to a radio button for selecting what property
-should be sorted on.
-- ``group_input`` expands to a radio button for selecting what property
-should be grouped on.
-The category search code above would expand to the following::
-<tr>
-<th>Category:</th>
-<td>
-<select name="category">
-<option value="">don't care</option>
-<option value="">------------</option>
-<option value="1">scipy</option>
-<option value="2">chaco</option>
-<option value="3">weave</option>
-</select>
-</td>
-<td><input type="checkbox" name=":columns" value="category"></td>
-<td><input type="radio" name=":sort0" value="category"></td>
-<td><input type="radio" name=":group0" value="category"></td>
-</tr>
-Adding category to the default view
-:::::::::::::::::::::::::::::::::::
-We can now add categories, add issues with categories, and search for
-issues based on categories. This is everything that we need to do;
-however, there is some more icing that we would like. I think the
-category of an issue is important enough that it should be displayed by
-default when listing all the issues.
-Unfortunately, this is a bit less obvious than the previous steps. The
-code defining how the issues look is in ``html/issue.index.html``. This
-is a large table with a form down at the bottom for redisplaying and so
-forth.
-Firstly we need to add an appropriate header to the start of the table::
-<th tal:condition="request/show/category">Category</th>
-The *condition* part of this statement is to avoid displaying the
-Category column if the user has selected not to see it.
-The rest of the table is a loop which will go through every issue that
-matches the display criteria. The loop variable is "i" - which means
-that every issue gets assigned to "i" in turn.
-The new part of code to display the category will look like this::
-<td tal:condition="request/show/category"
-tal:content="i/category"></td>
-The condition is the same as above: only display the condition when the
-user hasn't asked for it to be hidden. The next part is to set the
-content of the cell to be the category part of "i" - the current issue.
-Finally we have to edit ``html/page.html`` again. This time, we need to
-tell it that when the user clicks on "Unassigned Issues" or "All Issues",
-the category column should be included in the resulting list. If you
-scroll down the page file, you can see the links with lots of options.
-The option that we are interested in is the ``:columns=`` one which
-tells roundup which fields of the issue to display. Simply add
-"category" to that list and it all should work.
-Adding a time log to your issues
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-We want to log the dates and amount of time spent working on issues, and
-be able to give a summary of the total time spent on a particular issue.
-1. Add a new class to your tracker ``schema.py``::
-# storage for time logging
-timelog = Class(db, "timelog", period=Interval())
-Note that we automatically get the date of the time log entry
-creation through the standard property "creation".
-You will need to grant "Creation" permission to the users who are
-allowed to add timelog entries. You may do this with::
-db.security.addPermissionToRole('User', 'Create', 'timelog')
-db.security.addPermissionToRole('User', 'View', 'timelog')
-If users are also able to *edit* timelog entries, then also include::
-db.security.addPermissionToRole('User', 'Edit', 'timelog')
-.. index:: schema; example changes
-2. Link to the new class from your issue class (again, in
-``schema.py``)::
-issue = IssueClass(db, "issue",
-assignedto=Link("user"), keyword=Multilink("keyword"),
-priority=Link("priority"), status=Link("status"),
-times=Multilink("timelog"))
-the "times" property is the new link to the "timelog" class.
-3. We'll need to let people add in times to the issue, so in the web
-interface we'll have a new entry field. This is a special field
-because unlike the other fields in the ``issue.item`` template, it
-affects a different item (a timelog item) and not the template's
-item (an issue). We have a special syntax for form fields that affect
-items other than the template default item (see the cgi
-documentation on `special form variables`_). In particular, we add a
-field to capture a new timelog item's period::
-<tr>
-<th>Time Log</th>
-<td colspan=3><input type="text" name="timelog-1@period" />
-(enter as '3y 1m 4d 2:40:02' or parts thereof)
-</td>
-</tr>
-and another hidden field that links that new timelog item (new
-because it's marked as having id "-1") to the issue item. It looks
-like this::
-<input type="hidden" name="@link@times" value="timelog-1" />
-On submission, the "-1" timelog item will be created and assigned a
-real item id. The "times" property of the issue will have the new id
-added to it.
-The full entry will now look like this::
-<tr>
-<th>Time Log</th>
-<td colspan=3><input type="text" name="timelog-1@period" />
-(enter as '3y 1m 4d 2:40:02' or parts thereof)
-<input type="hidden" name="@link@times" value="timelog-1" />
-</td>
-</tr>
-4. We want to display a total of the timelog times that have been
-accumulated for an issue. To do this, we'll need to actually write
-some Python code, since it's beyond the scope of PageTemplates to
-perform such calculations. We do this by adding a module ``timespent.py``
-to the ``extensions`` directory in our tracker. The contents of this
-file is as follows::
-from roundup import date
-def totalTimeSpent(times):
-''' Call me with a list of timelog items (which have an
-Interval "period" property)
-'''
-total = date.Interval('0d')
-for time in times:
-total += time.period._value
-return total
-def init(instance):
-instance.registerUtil('totalTimeSpent', totalTimeSpent)
-We will now be able to access the ``totalTimeSpent`` function via the
-``utils`` variable in our templates, as shown in the next step.
-5. Display the timelog for an issue::
-<table class="otherinfo" tal:condition="context/times">
-<tr><th colspan="3" class="header">Time Log
-<tal:block
-tal:replace="python:utils.totalTimeSpent(context.times)" />
-</th></tr>
-<tr><th>Date</th><th>Period</th><th>Logged By</th></tr>
-<tr tal:repeat="time context/times">
-<td tal:content="time/creation"></td>
-<td tal:content="time/period"></td>
-<td tal:content="time/creator"></td>
-</tr>
-</table>
-I put this just above the Messages log in my issue display. Note our
-use of the ``totalTimeSpent`` method which will total up the times
-for the issue and return a new Interval. That will be automatically
-displayed in the template as text like "+ 1y 2:40" (1 year, 2 hours
-and 40 minutes).
-6. If you're using a persistent web server - ``roundup-server`` or
-``mod_wsgi`` for example - then you'll need to restart that to pick up
-the code changes. When that's done, you'll be able to use the new
-time logging interface.
-An extension of this modification attaches the timelog entries to any
-change message entered at the time of the timelog entry:
-A. Add a link to the timelog to the msg class in ``schema.py``:
-msg = FileClass(db, "msg",
-author=Link("user", do_journal='no'),
-recipients=Multilink("user", do_journal='no'),
-date=Date(),
-summary=String(),
-files=Multilink("file"),
-messageid=String(),
-inreplyto=String(),
-times=Multilink("timelog"))
-B. Add a new hidden field that links that new timelog item (new
-because it's marked as having id "-1") to the new message.
-The link is placed in ``issue.item.html`` in the same section that
-handles the timelog entry.
-It looks like this after this addition::
-<tr>
-<th>Time Log</th>
-<td colspan=3><input type="text" name="timelog-1@period" />
-(enter as '3y 1m 4d 2:40:02' or parts thereof)
-<input type="hidden" name="@link@times" value="timelog-1" />
-<input type="hidden" name="msg-1@link@times" value="timelog-1" />
-</td>
-</tr>
-The "times" property of the message will have the new id added to it.
-C. Add the timelog listing from step 5. to the ``msg.item.html`` template
-so that the timelog entry appears on the message view page. Note that
-the call to totalTimeSpent is not used here since there will only be one
-single timelog entry for each message.
-I placed it after the Date entry like this::
-<tr>
-<th i18n:translate="">Date:</th>
-<td tal:content="context/date"></td>
-</tr>
-</table>
-<table class="otherinfo" tal:condition="context/times">
-<tr><th colspan="3" class="header">Time Log</th></tr>
-<tr><th>Date</th><th>Period</th><th>Logged By</th></tr>
-<tr tal:repeat="time context/times">
-<td tal:content="time/creation"></td>
-<td tal:content="time/period"></td>
-<td tal:content="time/creator"></td>
-</tr>
-</table>
-<table class="messages">
-Tracking different types of issues
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Sometimes you will want to track different types of issues - developer,
-customer support, systems, sales leads, etc. A single Roundup tracker is
-able to support multiple types of issues. This example demonstrates adding
-a system support issue class to a tracker.
-1. Figure out what information you're going to want to capture. OK, so
-this is obvious, but sometimes it's better to actually sit down for a
-while and think about the schema you're going to implement.
-2. Add the new issue class to your tracker's ``schema.py``. Just after the
-"issue" class definition, add::
-# list our systems
-system = Class(db, "system", name=String(), order=Number())
-system.setkey("name")
-# store issues related to those systems
-support = IssueClass(db, "support",
-assignedto=Link("user"), keyword=Multilink("keyword"),
-status=Link("status"), deadline=Date(),
-affects=Multilink("system"))
-3. Copy the existing ``issue.*`` (item, search and index) templates in the
-tracker's ``html`` to ``support.*``. Edit them so they use the properties
-defined in the ``support`` class. Be sure to check for hidden form
-variables like "required" to make sure they have the correct set of
-required properties.
-4. Edit the modules in the ``detectors``, adding lines to their ``init``
-functions where appropriate. Look for ``audit`` and ``react`` registrations
-on the ``issue`` class, and duplicate them for ``support``.
-5. Create a new sidebar box for the new support class. Duplicate the
-existing issues one, changing the ``issue`` class name to ``support``.
-6. Re-start your tracker and start using the new ``support`` class.
-Optionally, you might want to restrict the users able to access this new
-class to just the users with a new "SysAdmin" Role. To do this, we add
-some security declarations::
-db.security.addPermissionToRole('SysAdmin', 'View', 'support')
-db.security.addPermissionToRole('SysAdmin', 'Create', 'support')
-db.security.addPermissionToRole('SysAdmin', 'Edit', 'support')
-You would then (as an "admin" user) edit the details of the appropriate
-users, and add "SysAdmin" to their Roles list.
-Alternatively, you might want to change the Edit/View permissions granted
-for the ``issue`` class so that it's only available to users with the "System"
-or "Developer" Role, and then the new class you're adding is available to
-all with the "User" Role.
-.. _external-authentication:
-Using External User Databases
------------------------------
-Using an external password validation source
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-.. note:: You will need to either have an "admin" user in your external
-password source *or* have one of your regular users have
-the Admin Role assigned. If you need to assign the Role *after*
-making the changes below, you may use the ``roundup-admin``
-program to edit a user's details.
-We have a centrally-managed password changing system for our users. This
-results in a UN*X passwd-style file that we use for verification of
-users. Entries in the file consist of ``name:password`` where the
-password is encrypted using the standard UN*X ``crypt()`` function (see
-the ``crypt`` module in your Python distribution). An example entry
-would be::
-admin:aamrgyQfDFSHw
-Each user of Roundup must still have their information stored in the Roundup
-database - we just use the passwd file to check their password. To do this, we
-need to override the standard ``verifyPassword`` method defined in
-``roundup.cgi.actions.LoginAction`` and register the new class. The
-following is added as ``externalpassword.py`` in the tracker ``extensions``
-directory::
-import os, crypt
-from roundup.cgi.actions import LoginAction
-class ExternalPasswordLoginAction(LoginAction):
-def verifyPassword(self, userid, password):
-'''Look through the file, line by line, looking for a
-name that matches.
-'''
-# get the user's username
-username = self.db.user.get(userid, 'username')
-# the passwords are stored in the "passwd.txt" file in the
-# tracker home
-file = os.path.join(self.db.config.TRACKER_HOME, 'passwd.txt')
-# see if we can find a match
-for ent in [line.strip().split(':') for line in
-open(file).readlines()]:
-if ent[0] == username:
-return crypt.crypt(password, ent[1][:2]) == ent[1]
-# user doesn't exist in the file
-return 0
-def init(instance):
-instance.registerAction('login', ExternalPasswordLoginAction)
-You should also remove the redundant password fields from the ``user.item``
-template.
-Using a UN*X passwd file as the user database
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-On some systems the primary store of users is the UN*X passwd file. It
-holds information on users such as their username, real name, password
-and primary user group.
-Roundup can use this store as its primary source of user information,
-but it needs additional information too - email address(es), roundup
-Roles, vacation flags, roundup hyperdb item ids, etc. Also, "retired"
-users must still exist in the user database, unlike some passwd files in
-which the users are removed when they no longer have access to a system.
-To make use of the passwd file, we therefore synchronise between the two
-user stores. We also use the passwd file to validate the user logins, as
-described in the previous example, `using an external password
-validation source`_. We keep the user lists in sync using a fairly
-simple script that runs once a day, or several times an hour if more
-immediate access is needed. In short, it:
-1. parses the passwd file, finding usernames, passwords and real names,
-2. compares that list to the current roundup user list:
-a. entries no longer in the passwd file are *retired*
-b. entries with mismatching real names are *updated*
-c. entries only exist in the passwd file are *created*
-3. send an email to administrators to let them know what's been done.
-The retiring and updating are simple operations, requiring only a call
-to ``retire()`` or ``set()``. The creation operation requires more
-information though - the user's email address and their Roundup Roles.
-We're going to assume that the user's email address is the same as their
-login name, so we just append the domain name to that. The Roles are
-determined using the passwd group identifier - mapping their UN*X group
-to an appropriate set of Roles.
-The script to perform all this, broken up into its main components, is
-as follows. Firstly, we import the necessary modules and open the
-tracker we're to work on::
-import sys, os, smtplib
-from roundup import instance, date
-# open the tracker
-tracker_home = sys.argv[1]
-tracker = instance.open(tracker_home)
-Next we read in the *passwd* file from the tracker home::
-# read in the users from the "passwd.txt" file
-file = os.path.join(tracker_home, 'passwd.txt')
-users = [x.strip().split(':') for x in open(file).readlines()]
-Handle special users (those to ignore in the file, and those who don't
-appear in the file)::
-# users to not keep ever, pre-load with the users I know aren't
-# "real" users
-ignore = ['ekmmon', 'bfast', 'csrmail']
-# users to keep - pre-load with the roundup-specific users
-keep = ['comment_pool', 'network_pool', 'admin', 'dev-team',
-'cs_pool', 'anonymous', 'system_pool', 'automated']
-Now we map the UN*X group numbers to the Roles that users should have::
-roles = {
-'501': 'User,Tech',  # tech
-'502': 'User',       # finance
-'503': 'User,CSR',   # customer service reps
-'504': 'User',       # sales
-'505': 'User',       # marketing
-}
-Now we do all the work. Note that the body of the script (where we have
-the tracker database open) is wrapped in a ``try`` / ``finally`` clause,
-so that we always close the database cleanly when we're finished. So, we
-now do all the work::
-# open the database
-db = tracker.open('admin')
-try:
-# store away messages to send to the tracker admins
-msg = []
-# loop over the users list read in from the passwd file
-for user,passw,uid,gid,real,home,shell in users:
-if user in ignore:
-# this user shouldn't appear in our tracker
-continue
-keep.append(user)
-try:
-# see if the user exists in the tracker
-uid = db.user.lookup(user)
-# yes, they do - now check the real name for correctness
-if real != db.user.get(uid, 'realname'):
-db.user.set(uid, realname=real)
-msg.append('FIX %s - %s'%(user, real))
-except KeyError:
-# nope, the user doesn't exist
-db.user.create(username=user, realname=real,
-address='%s@ekit-inc.com'%user, roles=roles[gid])
-msg.append('ADD %s - %s (%s)'%(user, real, roles[gid]))
-# now check that all the users in the tracker are also in our
-# "keep" list - retire those who aren't
-for uid in db.user.list():
-user = db.user.get(uid, 'username')
-if user not in keep:
-db.user.retire(uid)
-msg.append('RET %s'%user)
-# if we did work, then send email to the tracker admins
-if msg:
-# create the email
-msg = '''Subject: %s user database maintenance
-%s
-'''%(db.config.TRACKER_NAME, '\n'.join(msg))
-# send the email
-smtp = smtplib.SMTP(db.config.MAILHOST)
-addr = db.config.ADMIN_EMAIL
-smtp.sendmail(addr, addr, msg)
-# now we're done - commit the changes
-db.commit()
-finally:
-# always close the database cleanly
-db.close()
-And that's it!
-Using an LDAP database for user information
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-A script that reads users from an LDAP store using
-https://pypi.org/project/python-ldap/ and then compares the list to the users in the
-roundup user database would be pretty easy to write. You'd then have it run
-once an hour / day (or on demand if you can work that into your LDAP store
-workflow). See the example `Using a UN*X passwd file as the user database`_
-for more information about doing this.
-To authenticate off the LDAP store (rather than using the passwords in the
-Roundup user database) you'd use the same python-ldap module inside an
-extension to the cgi interface. You'd do this by overriding the method called
-``verifyPassword`` on the ``LoginAction`` class in your tracker's
-``extensions`` directory (see `using an external password validation
-source`_). The method is implemented by default as::
-def verifyPassword(self, userid, password):
-''' Verify the password that the user has supplied
-'''
-stored = self.db.user.get(self.userid, 'password')
-if password == stored:
-return 1
-if not password and not stored:
-return 1
-return 0
-So you could reimplement this as something like::
-def verifyPassword(self, userid, password):
-''' Verify the password that the user has supplied
-'''
-# look up some unique LDAP information about the user
-username = self.db.user.get(self.userid, 'username')
-# now verify the password supplied against the LDAP store
-Changes to Tracker Behaviour
-----------------------------
-.. index:: single: auditors; how to register (example)
-single: reactors; how to register (example)
-Preventing SPAM
-~~~~~~~~~~~~~~~
-The following detector code may be installed in your tracker's
-``detectors`` directory. It will block any messages being created that
-have HTML attachments (a very common vector for spam and phishing)
-and any messages that have more than 2 HTTP URLs in them. Just copy
-the following into ``detectors/anti_spam.py`` in your tracker::
-from roundup.exceptions import Reject
-def reject_html(db, cl, nodeid, newvalues):
-if newvalues['type'] == 'text/html':
-raise Reject('not allowed')
-def reject_manylinks(db, cl, nodeid, newvalues):
-content = newvalues['content']
-if content.count('http://') > 2:
-raise Reject('not allowed')
-def init(db):
-db.file.audit('create', reject_html)
-db.msg.audit('create', reject_manylinks)
-You may also wish to block image attachments if your tracker does not
-need that ability::
-if newvalues['type'].startswith('image/'):
-raise Reject('not allowed')
-Stop "nosy" messages going to people on vacation
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-When users go on vacation and set up vacation email bouncing, you'll
-start to see a lot of messages come back through Roundup "Fred is on
-vacation". Not very useful, and relatively easy to stop.
-1. add a "vacation" flag to your users::
-user = Class(db, "user",
-username=String(),   password=Password(),
-address=String(),    realname=String(),
-phone=String(),      organisation=String(),
-alternate_addresses=String(),
-roles=String(), queries=Multilink("query"),
-vacation=Boolean())
-2. So that users may edit the vacation flags, add something like the
-following to your ``user.item`` template::
-<tr>
-<th>On Vacation</th>
-<td tal:content="structure context/vacation/field">vacation</td>
-</tr>
-3. edit your detector ``nosyreactor.py`` so that the ``nosyreaction()``
-consists of::
-def nosyreaction(db, cl, nodeid, oldvalues):
-users = db.user
-messages = db.msg
-# send a copy of all new messages to the nosy list
-for msgid in determineNewMessages(cl, nodeid, oldvalues):
-try:
-# figure the recipient ids
-sendto = []
-seen_message = {}
-recipients = messages.get(msgid, 'recipients')
-for recipid in messages.get(msgid, 'recipients'):
-seen_message[recipid] = 1
-# figure the author's id, and indicate they've received
-# the message
-authid = messages.get(msgid, 'author')
-# possibly send the message to the author, as long as
-# they aren't anonymous
-if (db.config.MESSAGES_TO_AUTHOR == 'yes' and
-users.get(authid, 'username') != 'anonymous'):
-sendto.append(authid)
-seen_message[authid] = 1
-# now figure the nosy people who weren't recipients
-nosy = cl.get(nodeid, 'nosy')
-for nosyid in nosy:
-# Don't send nosy mail to the anonymous user (that
-# user shouldn't appear in the nosy list, but just
-# in case they do...)
-if users.get(nosyid, 'username') == 'anonymous':
-continue
-# make sure they haven't seen the message already
-if nosyid not in seen_message:
-# send it to them
-sendto.append(nosyid)
-recipients.append(nosyid)
-# generate a change note
-if oldvalues:
-note = cl.generateChangeNote(nodeid, oldvalues)
-else:
-note = cl.generateCreateNote(nodeid)
-# we have new recipients
-if sendto:
-# filter out the people on vacation
-sendto = [i for i in sendto
-if not users.get(i, 'vacation', 0)]
-# map userids to addresses
-sendto = [users.get(i, 'address') for i in sendto]
-# update the message's recipients list
-messages.set(msgid, recipients=recipients)
-# send the message
-cl.send_message(nodeid, msgid, note, sendto)
-except roundupdb.MessageSendError as message:
-raise roundupdb.DetectorError(message)
-Note that this is the standard nosy reaction code, with the small
-addition of::
-# filter out the people on vacation
-sendto = [i for i in sendto if not users.get(i, 'vacation', 0)]
-which filters out the users that have the vacation flag set to true.
-Adding in state transition control
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Sometimes tracker admins want to control the states to which users may
-move issues. You can do this by following these steps:
-1. make "status" a required variable. This is achieved by adding the
-following to the top of the form in the ``issue.item.html``
-template::
-<input type="hidden" name="@required" value="status">
-This will force users to select a status.
-2. add a Multilink property to the status class::
-stat = Class(db, "status", ... , transitions=Multilink('status'),
-...)
-and then edit the statuses already created, either:
-a. through the web using the class list -> status class editor, or
-b. using the ``roundup-admin`` "set" command.
-3. add an auditor module ``checktransition.py`` in your tracker's
-``detectors`` directory, for example::
-def checktransition(db, cl, nodeid, newvalues):
-''' Check that the desired transition is valid for the "status"
-property.
-'''
-if 'status' not in newvalues:
-return
-current = cl.get(nodeid, 'status')
-new = newvalues['status']
-if new == current:
-return
-ok = db.status.get(current, 'transitions')
-if new not in ok:
-raise ValueError('Status not allowed to move from "%s" to "%s"'%(
-db.status.get(current, 'name'), db.status.get(new, 'name')))
-def init(db):
-db.issue.audit('set', checktransition)
-4. in the ``issue.item.html`` template, change the status editing bit
-from::
-<th>Status</th>
-<td tal:content="structure context/status/menu">status</td>
-to::
-<th>Status</th>
-<td>
-<select tal:condition="context/id" name="status">
-<tal:block tal:define="ok context/status/transitions"
-tal:repeat="state db/status/list">
-<option tal:condition="python:state.id in ok"
-tal:attributes="
-value state/id;
-selected python:state.id == context.status.id"
-tal:content="state/name"></option>
-</tal:block>
-</select>
-<tal:block tal:condition="not:context/id"
-tal:replace="structure context/status/menu" />
-</td>
-which displays only the allowed status to transition to.
-Blocking issues that depend on other issues
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-We needed the ability to mark certain issues as "blockers" - that is,
-they can't be resolved until another issue (the blocker) they rely on is
-resolved. To achieve this:
-1. Create a new property on the ``issue`` class:
-``blockers=Multilink("issue")``. To do this, edit the definition of
-this class in your tracker's ``schema.py`` file. Change this::
-issue = IssueClass(db, "issue",
-assignedto=Link("user"), keyword=Multilink("keyword"),
-priority=Link("priority"), status=Link("status"))
-to this, adding the blockers entry::
-issue = IssueClass(db, "issue",
-blockers=Multilink("issue"),
-assignedto=Link("user"), keyword=Multilink("keyword"),
-priority=Link("priority"), status=Link("status"))
-2. Add the new ``blockers`` property to the ``issue.item.html`` edit
-page, using something like::
-<th>Waiting On</th>
-<td>
-<span tal:replace="structure python:context.blockers.field(showid=1,
-size=20)" />
-<span tal:replace="structure python:db.issue.classhelp('id,title',
-property='blockers')" />
-<span tal:condition="context/blockers"
-tal:repeat="blk context/blockers">
-<br>View: <a tal:attributes="href string:issue${blk/id}"
-tal:content="blk/id"></a>
-</span>
-</td>
-You'll need to fiddle with your item page layout to find an
-appropriate place to put it - I'll leave that fun part up to you.
-Just make sure it appears in the first table, possibly somewhere near
-the "superseders" field.
-3. Create a new detector module (see below) which enforces the rules:
-- issues may not be resolved if they have blockers
-- when a blocker is resolved, it's removed from issues it blocks
-The contents of the detector should be something like this::
-def blockresolution(db, cl, nodeid, newvalues):
-''' If the issue has blockers, don't allow it to be resolved.
-'''
-if nodeid is None:
-blockers = []
-else:
-blockers = cl.get(nodeid, 'blockers')
-blockers = newvalues.get('blockers', blockers)
-# don't do anything if there's no blockers or the status hasn't
-# changed
-if not blockers or 'status' not in newvalues:
-return
-# get the resolved state ID
-resolved_id = db.status.lookup('resolved')
-# format the info
-u = db.config.TRACKER_WEB
-s = ', '.join(['<a href="%sissue%s">%s</a>'%(
-u,id,id) for id in blockers])
-if len(blockers) == 1:
-s = 'issue %s is'%s
-else:
-s = 'issues %s are'%s
-# ok, see if we're trying to resolve
-if newvalues['status'] == resolved_id:
-raise ValueError("This issue can't be resolved until %s resolved."%s)
-def resolveblockers(db, cl, nodeid, oldvalues):
-''' When we resolve an issue that's a blocker, remove it from the
-blockers list of the issue(s) it blocks.
-'''
-newstatus = cl.get(nodeid,'status')
-# no change?
-if oldvalues.get('status', None) == newstatus:
-return
-resolved_id = db.status.lookup('resolved')
-# interesting?
-if newstatus != resolved_id:
-return
-# yes - find all the blocked issues, if any, and remove me from
-# their blockers list
-issues = cl.find(blockers=nodeid)
-for issueid in issues:
-blockers = cl.get(issueid, 'blockers')
-if nodeid in blockers:
-blockers.remove(nodeid)
-cl.set(issueid, blockers=blockers)
-def init(db):
-# might, in an obscure situation, happen in a create
-db.issue.audit('create', blockresolution)
-db.issue.audit('set', blockresolution)
-# can only happen on a set
-db.issue.react('set', resolveblockers)
-Put the above code in a file called "blockers.py" in your tracker's
-"detectors" directory.
-4. Finally, and this is an optional step, modify the tracker web page
-URLs so they filter out issues with any blockers. You do this by
-adding an additional filter on "blockers" for the value "-1". For
-example, the existing "Show All" link in the "page" template (in the
-tracker's "html" directory) looks like this::
-<a href="#"
-tal:attributes="href python:request.indexargs_url('issue', {
-'@sort': '-activity',
-'@group': 'priority',
-'@filter': 'status',
-'@columns': columns_showall,
-'@search_text': '',
-'status': status_notresolved,
-'@dispname': i18n.gettext('Show All'),
-})"
-i18n:translate="">Show All</a><br>
-modify it to add the "blockers" info to the URL (note, both the
-"@filter" *and* "blockers" values must be specified)::
-<a href="#"
-tal:attributes="href python:request.indexargs_url('issue', {
-'@sort': '-activity',
-'@group': 'priority',
-'@filter': 'status,blockers',
-'@columns': columns_showall,
-'@search_text': '',
-'status': status_notresolved,
-'blockers': '-1',
-'@dispname': i18n.gettext('Show All'),
-})"
-i18n:translate="">Show All</a><br>
-The above examples are line-wrapped on the trailing & and should
-be unwrapped.
-That's it. You should now be able to set blockers on your issues. Note
-that if you want to know whether an issue has any other issues dependent
-on it (i.e. it's in their blockers list) you can look at the journal
-history at the bottom of the issue page - look for a "link" event to
-another issue's "blockers" property.
-Add users to the nosy list based on the keyword
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Let's say we need the ability to automatically add users to the nosy
-list based
-on the occurance of a keyword. Every user should be allowed to edit their
-own list of keywords for which they want to be added to the nosy list.
-Below, we'll show that this change can be done with minimal
-understanding of the Roundup system, using only copy and paste.
-This requires three changes to the tracker: a change in the database to
-allow per-user recording of the lists of keywords for which he wants to
-be put on the nosy list, a change in the user view allowing them to edit
-this list of keywords, and addition of an auditor which updates the nosy
-list when a keyword is set.
-Adding the nosy keyword list
-::::::::::::::::::::::::::::
-The change to make in the database, is that for any user there should be a list
-of keywords for which he wants to be put on the nosy list. Adding a
-``Multilink`` of ``keyword`` seems to fullfill this. As such, all that has to
-be done is to add a new field to the definition of ``user`` within the file
-``schema.py``.  We will call this new field ``nosy_keywords``, and the updated
-definition of user will be::
-user = Class(db, "user",
-username=String(),   password=Password(),
-address=String(),    realname=String(),
-phone=String(),      organisation=String(),
-alternate_addresses=String(),
-queries=Multilink('query'), roles=String(),
-timezone=String(),
-nosy_keywords=Multilink('keyword'))
-Changing the user view to allow changing the nosy keyword list
-::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
-We want any user to be able to change the list of keywords for which
-he will by default be added to the nosy list. We choose to add this
-to the user view, as is generated by the file ``html/user.item.html``.
-We can easily
-see that the keyword field in the issue view has very similar editing
-requirements as our nosy keywords, both being lists of keywords. As
-such, we look for Keywords in ``issue.item.html``, and extract the
-associated parts from there. We add this to ``user.item.html`` at the
-bottom of the list of viewed items (i.e. just below the 'Alternate
-E-mail addresses' in the classic template)::
-<tr>
-<th>Nosy Keywords</th>
-<td>
-<span tal:replace="structure context/nosy_keywords/field" />
-<span tal:replace="structure python:db.keyword.classhelp(property='nosy_keywords')" />
-</td>
-</tr>
-Addition of an auditor to update the nosy list
-::::::::::::::::::::::::::::::::::::::::::::::
-The more difficult part is the logic to add
-the users to the nosy list when required.
-We choose to perform this action whenever the keywords on an
-item are set (this includes the creation of items).
-Here we choose to start out with a copy of the
-``detectors/nosyreaction.py`` detector, which we copy to the file
-``detectors/nosy_keyword_reaction.py``.
-This looks like a good start as it also adds users
-to the nosy list. A look through the code reveals that the
-``nosyreaction`` function actually sends the e-mail.
-We don't need this. Therefore, we can change the ``init`` function to::
-def init(db):
-db.issue.audit('create', update_kw_nosy)
-db.issue.audit('set', update_kw_nosy)
-After that, we rename the ``updatenosy`` function to ``update_kw_nosy``.
-The first two blocks of code in that function relate to setting
-``current`` to a combination of the old and new nosy lists. This
-functionality is left in the new auditor. The following block of
-code, which handled adding the assignedto user(s) to the nosy list in
-``updatenosy``, should be replaced by a block of code to add the
-interested users to the nosy list. We choose here to loop over all
-new keywords, than looping over all users,
-and assign the user to the nosy list when the keyword occurs in the user's
-``nosy_keywords``. The next part in ``updatenosy`` -- adding the author
-and/or recipients of a message to the nosy list -- is obviously not
-relevant here and is thus deleted from the new auditor. The last
-part, copying the new nosy list to ``newvalues``, can stay as is.
-This results in the following function::
-def update_kw_nosy(db, cl, nodeid, newvalues):
-'''Update the nosy list for changes to the keywords
-'''
-# nodeid will be None if this is a new node
-current = {}
-if nodeid is None:
-ok = ('new', 'yes')
-else:
-ok = ('yes',)
-# old node, get the current values from the node if they haven't
-# changed
-if 'nosy' not in newvalues:
-nosy = cl.get(nodeid, 'nosy')
-for value in nosy:
-if value not in current:
-current[value] = 1
-# if the nosy list changed in this transaction, init from the new value
-if 'nosy' in newvalues:
-nosy = newvalues.get('nosy', [])
-for value in nosy:
-if not db.hasnode('user', value):
-continue
-if value not in current:
-current[value] = 1
-# add users with keyword in nosy_keywords to the nosy list
-if 'keyword' in newvalues and newvalues['keyword'] is not None:
-keyword_ids = newvalues['keyword']
-for keyword in keyword_ids:
-# loop over all users,
-# and assign user to nosy when keyword in nosy_keywords
-for user_id in db.user.list():
-nosy_kw = db.user.get(user_id, "nosy_keywords")
-found = 0
-for kw in nosy_kw:
-if kw == keyword:
-found = 1
-if found:
-current[user_id] = 1
-# that's it, save off the new nosy list
-newvalues['nosy'] = list(current.keys())
-These two function are the only ones needed in the file.
-TODO: update this example to use the ``find()`` Class method.
-Caveats
-:::::::
-A few problems with the design here can be noted:
-Multiple additions
-When a user, after automatic selection, is manually removed
-from the nosy list, he is added to the nosy list again when the
-keyword list of the issue is updated. A better design might be
-to only check which keywords are new compared to the old list
-of keywords, and only add users when they have indicated
-interest on a new keyword.
-The code could also be changed to only trigger on the ``create()``
-event, rather than also on the ``set()`` event, thus only setting
-the nosy list when the issue is created.
-Scalability
-In the auditor, there is a loop over all users. For a site with
-only few users this will pose no serious problem; however, with
-many users this will be a serious performance bottleneck.
-A way out would be to link from the keywords to the users who
-selected these keywords as nosy keywords. This will eliminate the
-loop over all users. See the ``rev_multilink`` attribute to make
-this easier.
-Restricting updates that arrive by email
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Roundup supports multiple update methods:
-1. command line
-2. plain email
-3. pgp signed email
-4. web access
-in some cases you may need to prevent changes to properties by some of
-these methods. For example you can set up issues that are viewable
-only by people on the nosy list. So you must prevent unauthenticated
-changes to the nosy list.
-Since plain email can be easily forged, it does not provide sufficient
-authentication in this senario.
-To prevent this we can add a detector that audits the source of the
-transaction and rejects the update if it changes the nosy list.
-Create the detector (auditor) module and add it to the detectors
-directory of your tracker::
-from roundup import roundupdb, hyperdb
-from roundup.mailgw import Unauthorized
-def restrict_nosy_changes(db, cl, nodeid, newvalues):
-'''Do not permit changes to nosy via email.'''
-if 'nosy' not in newvalues:
-# the nosy field has not changed so no need to check.
-return
-if db.tx_Source in ['web', 'rest', 'xmlrpc', 'email-sig-openpgp', 'cli' ]:
-	   # if the source of the transaction is from an authenticated
-	   # source or a privileged process allow the transaction.
-	   # Other possible sources: 'email'
-	   return
-# otherwise raise an error
-raise Unauthorized( \
-	   'Changes to nosy property not allowed via %s for this issue.'%\
-tx_Source)
-def init(db):
-''' Install restrict_nosy_changes to run after other auditors.
-Allow initial creation email to set nosy.
-So don't execute: db.issue.audit('create', requestedbyauditor)
-Set priority to 110 to run this auditor after other auditors
-that can cause nosy to change.
-'''
-db.issue.audit('set', restrict_nosy_changes, 110)
-This detector (auditor) will prevent updates to the nosy field if it
-arrives by email. Since it runs after other auditors (due to the
-priority of 110), it will also prevent changes to the nosy field that
-are done by other auditors if triggered by an email.
-Note that db.tx_Source was not present in roundup versions before
-1.4.22, so you must be running a newer version to use this detector.
-Read the CHANGES.txt document in the roundup source code for further
-details on tx_Source.
-Changes to Security and Permissions
------------------------------------
-Restricting the list of users that are assignable to a task
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-1. In your tracker's ``schema.py``, create a new Role, say "Developer"::
-db.security.addRole(name='Developer', description='A developer')
-2. Just after that, create a new Permission, say "Fixer", specific to
-"issue"::
-p = db.security.addPermission(name='Fixer', klass='issue',
-description='User is allowed to be assigned to fix issues')
-3. Then assign the new Permission to your "Developer" Role::
-db.security.addPermissionToRole('Developer', p)
-4. In the issue item edit page (``html/issue.item.html`` in your tracker
-directory), use the new Permission in restricting the "assignedto"
-list::
-<select name="assignedto">
-<option value="-1">- no selection -</option>
-<tal:block tal:repeat="user db/user/list">
-<option tal:condition="python:user.hasPermission(
-'Fixer', context._classname)"
-tal:attributes="
-value user/id;
-selected python:user.id == context.assignedto"
-tal:content="user/realname"></option>
-</tal:block>
-</select>
-For extra security, you may wish to setup an auditor to enforce the
-Permission requirement (install this as ``assignedtoFixer.py`` in your
-tracker ``detectors`` directory)::
-def assignedtoMustBeFixer(db, cl, nodeid, newvalues):
-''' Ensure the assignedto value in newvalues is used with the
-Fixer Permission
-'''
-if 'assignedto' not in newvalues:
-# don't care
-return
-# get the userid
-userid = newvalues['assignedto']
-if not db.security.hasPermission('Fixer', userid, cl.classname):
-raise ValueError('You do not have permission to edit %s'%cl.classname)
-def init(db):
-db.issue.audit('set', assignedtoMustBeFixer)
-db.issue.audit('create', assignedtoMustBeFixer)
-So now, if an edit action attempts to set "assignedto" to a user that
-doesn't have the "Fixer" Permission, the error will be raised.
-Users may only edit their issues
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-In this case, users registering themselves are granted Provisional
-access, meaning they
-have access to edit the issues they submit, but not others. We create a new
-Role called "Provisional User" which is granted to newly-registered users,
-and has limited access. One of the Permissions they have is the new "Edit
-Own" on issues (regular users have "Edit".)
-First up, we create the new Role and Permission structure in
-``schema.py``::
-#
-# New users not approved by the admin
-#
-db.security.addRole(name='Provisional User',
-description='New user registered via web or email')
-# These users need to be able to view and create issues but only edit
-# and view their own
-db.security.addPermissionToRole('Provisional User', 'Create', 'issue')
-def own_issue(db, userid, itemid):
-'''Determine whether the userid matches the creator of the issue.'''
-return userid == db.issue.get(itemid, 'creator')
-p = db.security.addPermission(name='Edit', klass='issue',
-check=own_issue, description='Can only edit own issues')
-db.security.addPermissionToRole('Provisional User', p)
-p = db.security.addPermission(name='View', klass='issue',
-check=own_issue, description='Can only view own issues')
-db.security.addPermissionToRole('Provisional User', p)
-# This allows the interface to get the names of the properties
-# in the issue. Used for selecting sorting and grouping
-# on the index page.
-p = db.security.addPermission(name='Search', klass='issue')
-db.security.addPermissionToRole ('Provisional User', p)
-# Assign the Permissions for issue-related classes
-for cl in 'file', 'msg', 'query', 'keyword':
-db.security.addPermissionToRole('Provisional User', 'View', cl)
-db.security.addPermissionToRole('Provisional User', 'Edit', cl)
-db.security.addPermissionToRole('Provisional User', 'Create', cl)
-for cl in 'priority', 'status':
-db.security.addPermissionToRole('Provisional User', 'View', cl)
-# and give the new users access to the web and email interface
-db.security.addPermissionToRole('Provisional User', 'Web Access')
-db.security.addPermissionToRole('Provisional User', 'Email Access')
-# make sure they can view & edit their own user record
-def own_record(db, userid, itemid):
-'''Determine whether the userid matches the item being accessed.'''
-return userid == itemid
-p = db.security.addPermission(name='View', klass='user', check=own_record,
-description="User is allowed to view their own user details")
-db.security.addPermissionToRole('Provisional User', p)
-p = db.security.addPermission(name='Edit', klass='user', check=own_record,
-description="User is allowed to edit their own user details")
-db.security.addPermissionToRole('Provisional User', p)
-Then, in ``config.ini``, we change the Role assigned to newly-registered
-users, replacing the existing ``'User'`` values::
-[main]
-...
-new_web_user_roles = Provisional User
-new_email_user_roles = Provisional User
-All users may only view and edit issues, files and messages they create
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Replace the standard "classic" tracker View and Edit Permission assignments
-for the "issue", "file" and "msg" classes with the following::
-def checker(klass):
-def check(db, userid, itemid, klass=klass):
-return db.getclass(klass).get(itemid, 'creator') == userid
-return check
-for cl in 'issue', 'file', 'msg':
-p = db.security.addPermission(name='View', klass=cl,
-check=checker(cl),
-description='User can view only if creator.')
-db.security.addPermissionToRole('User', p)
-p = db.security.addPermission(name='Edit', klass=cl,
-check=checker(cl),
-description='User can edit only if creator.')
-db.security.addPermissionToRole('User', p)
-db.security.addPermissionToRole('User', 'Create', cl)
-# This allows the interface to get the names of the properties
-# in the issue. Used for selecting sorting and grouping
-# on the index page.
-p = db.security.addPermission(name='Search', klass='issue')
-db.security.addPermissionToRole ('User', p)
-Moderating user registration
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-You could set up new-user moderation in a public tracker by:
-1. creating a new highly-restricted user role "Pending",
-2. set the config new_web_user_roles and/or new_email_user_roles to that
-role,
-3. have an auditor that emails you when new users are created with that
-role using roundup.mailer
-4. edit the role to "User" for valid users.
-Some simple javascript might help in the last step. If you have high volume
-you could search for all currently-Pending users and do a bulk edit of all
-their roles at once (again probably with some simple javascript help).
-Changes to the Web User Interface
----------------------------------
-Adding action links to the index page
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Add a column to the ``item.index.html`` template.
-Resolving the issue::
-<a tal:attributes="href
-string:issue${i/id}?:status=resolved&:action=edit">resolve</a>
-"Take" the issue::
-<a tal:attributes="href
-string:issue${i/id}?:assignedto=${request/user/id}&:action=edit">take</a>
-... and so on.
-Colouring the rows in the issue index according to priority
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-A simple ``tal:attributes`` statement will do the bulk of the work here. In
-the ``issue.index.html`` template, add this to the ``<tr>`` that
-displays the rows of data::
-<tr tal:attributes="class string:priority-${i/priority/plain}">
-and then in your stylesheet (``style.css``) specify the colouring for the
-different priorities, as follows::
-tr.priority-critical td {
-background-color: red;
-}
-tr.priority-urgent td {
-background-color: orange;
-}
-and so on, with far less offensive colours :)
-Editing multiple items in an index view
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-To edit the status of all items in the item index view, edit the
-``issue.index.html``:
-1. add a form around the listing table (separate from the existing
-index-page form), so at the top it reads::
-<form method="POST" tal:attributes="action request/classname">
-<table class="list">
-and at the bottom of that table::
-</table>
-</form
-making sure you match the ``</table>`` from the list table, not the
-navigation table or the subsequent form table.
-2. in the display for the issue property, change::
-<td tal:condition="request/show/status"
-tal:content="python:i.status.plain() or default">&nbsp;</td>
-to::
-<td tal:condition="request/show/status"
-tal:content="structure i/status/field">&nbsp;</td>
-this will result in an edit field for the status property.
-3. after the ``tal:block`` which lists the index items (marked by
-``tal:repeat="i batch"``) add a new table row::
-<tr>
-<td tal:attributes="colspan python:len(request.columns)">
-<input name="@csrf" type="hidden"
-tal:attributes="value python:utils.anti_csrf_nonce()">
-<input type="submit" value=" Save Changes ">
-<input type="hidden" name="@action" value="edit">
-<tal:block replace="structure request/indexargs_form" />
-</td>
-</tr>
-which gives us a submit button, indicates that we are performing an
-edit on any changed statuses, and provides a defense against cross
-site request forgery attacks.
-The final ``tal:block`` will make sure that the current index view
-parameters (filtering, columns, etc) will be used in rendering the
-next page (the results of the editing).
-Displaying only message summaries in the issue display
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Alter the ``issue.item`` template section for messages to::
-<table class="messages" tal:condition="context/messages">
-<tr><th colspan="5" class="header">Messages</th></tr>
-<tr tal:repeat="msg context/messages">
-<td><a tal:attributes="href string:msg${msg/id}"
-tal:content="string:msg${msg/id}"></a></td>
-<td tal:content="msg/author">author</td>
-<td class="date" tal:content="msg/date/pretty">date</td>
-<td tal:content="msg/summary">summary</td>
-<td>
-<a tal:attributes="href string:?@remove@messages=${msg/id}&@action=edit">
-remove</a>
-</td>
-</tr>
-</table>
-Enabling display of either message summaries or the entire messages
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-This is pretty simple - all we need to do is copy the code from the
-example `displaying only message summaries in the issue display`_ into
-our template alongside the summary display, and then introduce a switch
-that shows either the one or the other. We'll use a new form variable,
-``@whole_messages`` to achieve this::
-<table class="messages" tal:condition="context/messages">
-<tal:block tal:condition="not:request/form/@whole_messages/value | python:0">
-<tr><th colspan="3" class="header">Messages</th>
-<th colspan="2" class="header">
-<a href="?@whole_messages=yes">show entire messages</a>
-</th>
-</tr>
-<tr tal:repeat="msg context/messages">
-<td><a tal:attributes="href string:msg${msg/id}"
-tal:content="string:msg${msg/id}"></a></td>
-<td tal:content="msg/author">author</td>
-<td class="date" tal:content="msg/date/pretty">date</td>
-<td tal:content="msg/summary">summary</td>
-<td>
-<a tal:attributes="href string:?@remove@messages=${msg/id}&@action=edit">remove</a>
-</td>
-</tr>
-</tal:block>
-<tal:block tal:condition="request/form/@whole_messages/value | python:0">
-<tr><th colspan="2" class="header">Messages</th>
-<th class="header">
-<a href="?@whole_messages=">show only summaries</a>
-</th>
-</tr>
-<tal:block tal:repeat="msg context/messages">
-<tr>
-<th tal:content="msg/author">author</th>
-<th class="date" tal:content="msg/date/pretty">date</th>
-<th style="text-align: right">
-(<a tal:attributes="href string:?@remove@messages=${msg/id}&@action=edit">remove</a>)
-</th>
-</tr>
-<tr><td colspan="3" tal:content="msg/content"></td></tr>
-</tal:block>
-</tal:block>
-</table>
-Setting up a "wizard" (or "druid") for controlled adding of issues
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-1. Set up the page templates you wish to use for data input. My wizard
-is going to be a two-step process: first figuring out what category
-of issue the user is submitting, and then getting details specific to
-that category. The first page includes a table of help, explaining
-what the category names mean, and then the core of the form::
-<form method="POST" onSubmit="return submit_once()"
-enctype="multipart/form-data">
-<input name="@csrf" type="hidden"
-tal:attributes="value python:utils.anti_csrf_nonce()">
-<input type="hidden" name="@template" value="add_page1">
-<input type="hidden" name="@action" value="page1_submit">
-<strong>Category:</strong>
-<tal:block tal:replace="structure context/category/menu" />
-<input type="submit" value="Continue">
-</form>
-The next page has the usual issue entry information, with the
-addition of the following form fragments::
-<form method="POST" onSubmit="return submit_once()"
-enctype="multipart/form-data"
-tal:condition="context/is_edit_ok"
-tal:define="cat request/form/category/value">
-<input name="@csrf" type="hidden"
-tal:attributes="value python:utils.anti_csrf_nonce()">
-<input type="hidden" name="@template" value="add_page2">
-<input type="hidden" name="@required" value="title">
-<input type="hidden" name="category" tal:attributes="value cat">
-.
-.
-.
-</form>
-Note that later in the form, I use the value of "cat" to decide which
-form elements should be displayed. For example::
-<tal:block tal:condition="python:cat in '6 10 13 14 15 16 17'.split()">
-<tr>
-<th>Operating System</th>
-<td tal:content="structure context/os/field"></td>
-</tr>
-<tr>
-<th>Web Browser</th>
-<td tal:content="structure context/browser/field"></td>
-</tr>
-</tal:block>
-... the above section will only be displayed if the category is one
-of 6, 10, 13, 14, 15, 16 or 17.
-3. Determine what actions need to be taken between the pages - these are
-usually to validate user choices and determine what page is next. Now encode
-those actions in a new ``Action`` class (see `defining new web actions`_)::
-from roundup.cgi.actions import Action
-class Page1SubmitAction(Action):
-def handle(self):
-''' Verify that the user has selected a category, and then move
-on to page 2.
-'''
-category = self.form['category'].value
-if category == '-1':
-self.client.add_error_message('You must select a category of report')
-return
-# everything's ok, move on to the next page
-self.client.template = 'add_page2'
-def init(instance):
-instance.registerAction('page1_submit', Page1SubmitAction)
-4. Use the usual "new" action as the ``@action`` on the final page, and
-you're done (the standard context/submit method can do this for you).
-Silent Submit
-~~~~~~~~~~~~~
-When working on an issue, most of the time the people on the nosy list
-need to be notified of changes. There are cases where a user wants to
-add a comment to an issue and not bother other users on the nosy
-list.
-This feature is called Silent Submit because it allows the user to
-silently modify an issue and not tell anyone.
-There are several parts to this change. The main activity part
-involves editing the stock detectors/nosyreaction.py file in your
-tracker. Insert the following lines near the top of the nosyreaction
-function::
-# Did user click button to do a silent change?
-try:
-if db.web['submit'] == "silent_change":
-return
-except (AttributeError, KeyError) as err:
-# The web attribute or submit key don't exist.
-# That's fine. We were probably triggered by an email
-# or cli based change.
-pass
-This checks the submit button to see if it is the silent type. If there
-are exceptions trying to make that determination they are ignored and
-processing continues. You may wonder how db.web gets set. This is done
-by creating an extension. Add the file extensions/edit.py with
-this content::
-from roundup.cgi.actions import EditItemAction
-class Edit2Action(EditItemAction):
-def handle(self):
-self.db.web = {}  # create the dict
-# populate the dict by getting the value of the submit_button
-# element from the form.
-self.db.web['submit'] = self.form['submit_button'].value
-# call the core EditItemAction to process the edit.
-EditItemAction.handle(self)
-def init(instance):
-'''Override the default edit action with this new version'''
-instance.registerAction('edit', Edit2Action)
-This code is a wrapper for the roundup EditItemAction. It checks the
-form's submit button to save the value element. The rest of the changes
-needed for the Silent Submit feature involves editing
-html/issue.item.html to add the silent submit button. In
-the stock issue.item.html the submit button is on a line that contains
-"submit button". Replace that line with something like the following::
-	    <input type="submit" name="submit_button"
-		   tal:condition="context/is_edit_ok"
-		   value="Submit Changes">&nbsp;
-	    <button type="submit" name="submit_button"
-		    tal:condition="context/is_edit_ok"
-		    title="Click this to submit but not send nosy email."
-		    value="silent_change" i18n:translate="">
-	      Silent Change</button>
-Note the difference in the value attribute for the two submit buttons.
-The value "silent_change" in the button specification must match the
-string in the nosy reaction function.
-Debugging Trackers
-==================
-There are three switches in tracker configs that turn on debugging in
-Roundup:
-1. web :: debug
-2. mail :: debug
-3. logging :: level
-See the config.ini file or the `tracker configuration`_ section above for
-more information.
-Additionally, the ``roundup-server.py`` script has its own debugging mode
-in which it reloads edited templates immediately when they are changed,
-rather than requiring a web server restart.
 .. _`design documentation`: design.html
 .. _`developer's guide`: developers.html
 .. _`rest interface documentation`: rest.html#programming-the-rest-api
+.. _change the rate limiting method: rest.html#creating-custom-rate-limits
 .. _`directions in the rest interface documentation`: rest.html#enabling-the-rest-api
 .. _`xmlrpc interface documentation`: xmlrpc.html#through-roundup
 .. _`zxcvbn`: https://github.com/dwolfhub/zxcvbn-python

Mercurial > p > roundup > code

comparison doc/customizing.txt @ 7464:82bbb95e5690 issue2550923_computed_property