Mercurial Repository: p/roundup/code: doc/design.txt comparison

comparison doc/design.txt @ 907:38a74d1351c5

documentation updates

author	Richard Jones <richard@users.sourceforge.net>
date	Mon, 29 Jul 2002 00:54:41 +0000
parents
children	299f4890427d

comparison

equal deleted inserted replaced

-:9a6a193fb62a
+:38a74d1351c5
+========================================================
+Roundup - An Issue-Tracking System for Knowledge Workers
+========================================================
+:Authors: Ka-Ping Yee (original__), Richard Jones (implementation)
+__ spec.html
+.. contents::
+Introduction
+---------------
+This document presents a description of the components
+of the Roundup system and specifies their interfaces and
+behaviour in sufficient detail to guide an implementation.
+For the philosophy and rationale behind the Roundup design,
+see the first-round Software Carpentry submission for Roundup.
+This document fleshes out that design as well as specifying
+interfaces so that the components can be developed separately.
+The Layer Cake
+-----------------
+Lots of software design documents come with a picture of
+a cake.  Everybody seems to like them.  I also like cakes
+(i think they are tasty).  So i, too, shall include
+a picture of a cake here::
+_________________________________________________________________________
+|  E-mail Client   |   Web Browser   |   Detector Scripts   |    Shell    |
+|------------------+-----------------+----------------------+-------------|
+|   E-mail User    |    Web User     |      Detector        |   Command   |
+|-------------------------------------------------------------------------|
+|                         Roundup Database Layer                          |
+|-------------------------------------------------------------------------|
+|                          Hyperdatabase Layer                            |
+|-------------------------------------------------------------------------|
+|                             Storage Layer                               |
+-------------------------------------------------------------------------
+The colourful parts of the cake are part of our system;
+the faint grey parts of the cake are external components.
+I will now proceed to forgo all table manners and
+eat from the bottom of the cake to the top.  You may want
+to stand back a bit so you don't get covered in crumbs.
+Hyperdatabase
+-------------
+The lowest-level component to be implemented is the hyperdatabase.
+The hyperdatabase is intended to be
+a flexible data store that can hold configurable data in
+records which we call nodes.
+The hyperdatabase is implemented on top of the storage layer,
+an external module for storing its data.  The storage layer could
+be a third-party RDBMS; for a "batteries-included" distribution,
+implementing the hyperdatabase on the standard bsddb
+module is suggested.
+Dates and Date Arithmetic
+~~~~~~~~~~~~~~~~~~~~~~~~~
+Before we get into the hyperdatabase itself, we need a
+way of handling dates.  The hyperdatabase module provides
+Timestamp objects for
+representing date-and-time stamps and Interval objects for
+representing date-and-time intervals.
+As strings, date-and-time stamps are specified with
+the date in international standard format
+(``yyyy-mm-dd``)
+joined to the time (``hh:mm:ss``)
+by a period "``.``".  Dates in
+this form can be easily compared and are fairly readable
+when printed.  An example of a valid stamp is
+"``2000-06-24.13:03:59``".
+We'll call this the "full date format".  When Timestamp objects are
+printed as strings, they appear in the full date format with
+the time always given in GMT.  The full date format is always
+exactly 19 characters long.
+For user input, some partial forms are also permitted:
+the whole time or just the seconds may be omitted; and the whole date
+may be omitted or just the year may be omitted.  If the time is given,
+the time is interpreted in the user's local time zone.
+The Date constructor takes care of these conversions.
+In the following examples, suppose that ``yyyy`` is the current year,
+``mm`` is the current month, and ``dd`` is the current
+day of the month; and suppose that the user is on Eastern Standard Time.
+-   "2000-04-17" means <Date 2000-04-17.00:00:00>
+-   "01-25" means <Date yyyy-01-25.00:00:00>
+-   "2000-04-17.03:45" means <Date 2000-04-17.08:45:00>
+-   "08-13.22:13" means <Date yyyy-08-14.03:13:00>
+-   "11-07.09:32:43" means <Date yyyy-11-07.14:32:43>
+-   "14:25" means
+-   <Date yyyy-mm-dd.19:25:00>
+-   "8:47:11" means
+-   <Date yyyy-mm-dd.13:47:11>
+-   the special date "." means "right now"
+Date intervals are specified using the suffixes
+"y", "m", and "d".  The suffix "w" (for "week") means 7 days.
+Time intervals are specified in hh:mm:ss format (the seconds
+may be omitted, but the hours and minutes may not).
+-   "3y" means three years
+-   "2y 1m" means two years and one month
+-   "1m 25d" means one month and 25 days
+-   "2w 3d" means two weeks and three days
+-   "1d 2:50" means one day, two hours, and 50 minutes
+-   "14:00" means 14 hours
+-   "0:04:33" means four minutes and 33 seconds
+The Date class should understand simple date expressions of the form
+*stamp* ``+`` *interval* and *stamp* ``-`` *interval*.
+When adding or subtracting intervals involving months or years, the
+components are handled separately.  For example, when evaluating
+"``2000-06-25 + 1m 10d``", we first add one month to
+get 2000-07-25, then add 10 days to get
+2000-08-04 (rather than trying to decide whether
+1m 10d means 38 or 40 or 41 days).
+Here is an outline of the Date and Interval classes::
+class Date:
+def __init__(self, spec, offset):
+"""Construct a date given a specification and a time zone offset.
+'spec' is a full date or a partial form, with an optional
+added or subtracted interval.  'offset' is the local time
+zone offset from GMT in hours.
+"""
+def __add__(self, interval):
+"""Add an interval to this date to produce another date."""
+def __sub__(self, interval):
+"""Subtract an interval from this date to produce another date."""
+def __cmp__(self, other):
+"""Compare this date to another date."""
+def __str__(self):
+"""Return this date as a string in the yyyy-mm-dd.hh:mm:ss format."""
+def local(self, offset):
+"""Return this date as yyyy-mm-dd.hh:mm:ss in a local time zone."""
+class Interval:
+def __init__(self, spec):
+"""Construct an interval given a specification."""
+def __cmp__(self, other):
+"""Compare this interval to another interval."""
+def __str__(self):
+"""Return this interval as a string."""
+Here are some examples of how these classes would behave in practice.
+For the following examples, assume that we are on Eastern Standard
+Time and the current local time is 19:34:02 on 25 June 2000::
+>>> Date(".")
+<Date 2000-06-26.00:34:02>
+>>> _.local(-5)
+"2000-06-25.19:34:02"
+>>> Date(". + 2d")
+<Date 2000-06-28.00:34:02>
+>>> Date("1997-04-17", -5)
+<Date 1997-04-17.00:00:00>
+>>> Date("01-25", -5)
+<Date 2000-01-25.00:00:00>
+>>> Date("08-13.22:13", -5)
+<Date 2000-08-14.03:13:00>
+>>> Date("14:25", -5)
+<Date 2000-06-25.19:25:00>
+>>> Interval("  3w  1  d  2:00")
+<Interval 22d 2:00>
+>>> Date(". + 2d") - Interval("3w")
+<Date 2000-06-07.00:34:02>
+Nodes and Classes
+~~~~~~~~~~~~~~~~~
+Nodes contain data in properties.  To Python, these
+properties are presented as the key-value pairs of a dictionary.
+Each node belongs to a class which defines the names
+and types of its properties.  The database permits the creation
+and modification of classes as well as nodes.
+Identifiers and Designators
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Each node has a numeric identifier which is unique among
+nodes in its class.  The nodes are numbered sequentially
+within each class in order of creation, starting from 1.
+The designator
+for a node is a way to identify a node in the database, and
+consists of the name of the node's class concatenated with
+the node's numeric identifier.
+For example, if "spam" and "eggs" are classes, the first
+node created in class "spam" has id 1 and designator "spam1".
+The first node created in class "eggs" also has id 1 but has
+the distinct designator "eggs1".  Node designators are
+conventionally enclosed in square brackets when mentioned
+in plain text.  This permits a casual mention of, say,
+"[patch37]" in an e-mail message to be turned into an active
+hyperlink.
+Property Names and Types
+~~~~~~~~~~~~~~~~~~~~~~~~
+Property names must begin with a letter.
+A property may be one of five basic types:
+- String properties are for storing arbitrary-length strings.
+- Boolean properties are for storing true/false, or yes/no values.
+- Number properties are for storing numeric values.
+- Date properties store date-and-time stamps.
+Their values are Timestamp objects.
+- A Link property refers to a single other node
+selected from a specified class.  The class is part of the property;
+the value is an integer, the id of the chosen node.
+- A Multilink property refers to possibly many nodes
+in a specified class.  The value is a list of integers.
+*None* is also a permitted value for any of these property
+types.  An attempt to store None into a Multilink property stores an empty list.
+A property that is not specified will return as None from a *get*
+operation.
+Hyperdb Interface Specification
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The hyperdb module provides property objects to designate
+the different kinds of properties.  These objects are used when
+specifying what properties belong in classes::
+class String:
+def __init__(self, indexme='no'):
+"""An object designating a String property."""
+class Boolean:
+def __init__(self):
+"""An object designating a Boolean property."""
+class Number:
+def __init__(self):
+"""An object designating a Number property."""
+class Date:
+def __init__(self):
+"""An object designating a Date property."""
+class Link:
+def __init__(self, classname, do_journal='yes'):
+"""An object designating a Link property that links to
+nodes in a specified class.
+If the do_journal argument is not 'yes' then changes to
+the property are not journalled in the linked node.
+"""
+class Multilink:
+def __init__(self, classname, do_journal='yes'):
+"""An object designating a Multilink property that links
+to nodes in a specified class.
+If the do_journal argument is not 'yes' then changes to
+the property are not journalled in the linked node(s).
+"""
+Here is the interface provided by the hyperdatabase::
+class Database:
+"""A database for storing records containing flexible data types."""
+def __init__(self, storagelocator, journaltag):
+"""Open a hyperdatabase given a specifier to some storage.
+The meaning of 'storagelocator' depends on the particular
+implementation of the hyperdatabase.  It could be a file name,
+a directory path, a socket descriptor for a connection to a
+database over the network, etc.
+The 'journaltag' is a token that will be attached to the journal
+entries for any edits done on the database.  If 'journaltag' is
+None, the database is opened in read-only mode: the Class.create(),
+Class.set(), and Class.retire() methods are disabled.
+"""
+def __getattr__(self, classname):
+"""A convenient way of calling self.getclass(classname)."""
+def getclasses(self):
+"""Return a list of the names of all existing classes."""
+def getclass(self, classname):
+"""Get the Class object representing a particular class.
+If 'classname' is not a valid class name, a KeyError is raised.
+"""
+class Class:
+"""The handle to a particular class of nodes in a hyperdatabase."""
+def __init__(self, db, classname, **properties):
+"""Create a new class with a given name and property specification.
+'classname' must not collide with the name of an existing class,
+or a ValueError is raised.  The keyword arguments in 'properties'
+must map names to property objects, or a TypeError is raised.
+"""
+# Editing nodes:
+def create(self, **propvalues):
+"""Create a new node of this class and return its id.
+The keyword arguments in 'propvalues' map property names to values.
+The values of arguments must be acceptable for the types of their
+corresponding properties or a TypeError is raised.  If this class
+has a key property, it must be present and its value must not
+collide with other key strings or a ValueError is raised.  Any other
+properties on this class that are missing from the 'propvalues'
+dictionary are set to None.  If an id in a link or multilink
+property does not refer to a valid node, an IndexError is raised.
+"""
+def get(self, nodeid, propname):
+"""Get the value of a property on an existing node of this class.
+'nodeid' must be the id of an existing node of this class or an
+IndexError is raised.  'propname' must be the name of a property
+of this class or a KeyError is raised.
+"""
+def set(self, nodeid, **propvalues):
+"""Modify a property on an existing node of this class.
+'nodeid' must be the id of an existing node of this class or an
+IndexError is raised.  Each key in 'propvalues' must be the name
+of a property of this class or a KeyError is raised.  All values
+in 'propvalues' must be acceptable types for their corresponding
+properties or a TypeError is raised.  If the value of the key
+property is set, it must not collide with other key strings or a
+ValueError is raised.  If the value of a Link or Multilink
+property contains an invalid node id, a ValueError is raised.
+"""
+def retire(self, nodeid):
+"""Retire a node.
+The properties on the node remain available from the get() method,
+and the node's id is never reused.  Retired nodes are not returned
+by the find(), list(), or lookup() methods, and other nodes may
+reuse the values of their key properties.
+"""
+def history(self, nodeid):
+"""Retrieve the journal of edits on a particular node.
+'nodeid' must be the id of an existing node of this class or an
+IndexError is raised.
+The returned list contains tuples of the form
+(date, tag, action, params)
+'date' is a Timestamp object specifying the time of the change and
+'tag' is the journaltag specified when the database was opened.
+'action' may be:
+'create' or 'set' -- 'params' is a dictionary of property values
+'link' or 'unlink' -- 'params' is (classname, nodeid, propname)
+'retire' -- 'params' is None
+"""
+# Locating nodes:
+def setkey(self, propname):
+"""Select a String property of this class to be the key property.
+'propname' must be the name of a String property of this class or
+None, or a TypeError is raised.  The values of the key property on
+all existing nodes must be unique or a ValueError is raised.
+"""
+def getkey(self):
+"""Return the name of the key property for this class or None."""
+def lookup(self, keyvalue):
+"""Locate a particular node by its key property and return its id.
+If this class has no key property, a TypeError is raised.  If the
+'keyvalue' matches one of the values for the key property among
+the nodes in this class, the matching node's id is returned;
+otherwise a KeyError is raised.
+"""
+def find(self, propname, nodeid):
+"""Get the ids of nodes in this class which link to a given node.
+'propname' must be the name of a property in this class, or a
+KeyError is raised.  That property must be a Link or Multilink
+property, or a TypeError is raised.  'nodeid' must be the id of
+an existing node in the class linked to by the given property,
+or an IndexError is raised.
+"""
+def list(self):
+"""Return a list of the ids of the active nodes in this class."""
+def count(self):
+"""Get the number of nodes in this class.
+If the returned integer is 'numnodes', the ids of all the nodes
+in this class run from 1 to numnodes, and numnodes+1 will be the
+id of the next node to be created in this class.
+"""
+# Manipulating properties:
+def getprops(self):
+"""Return a dictionary mapping property names to property objects."""
+def addprop(self, **properties):
+"""Add properties to this class.
+The keyword arguments in 'properties' must map names to property
+objects, or a TypeError is raised.  None of the keys in 'properties'
+may collide with the names of existing properties, or a ValueError
+is raised before any properties have been added.
+"""
+TODO: additional methods
+Hyperdatabase Implementations
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Hyperdatabase implementations exist to create the interface described in the
+`hyperdb interface specification`_
+over an existing storage mechanism. Examples are relational databases,
+\*dbm key-value databases, and so on.
+TODO: finish
+Application Example
+~~~~~~~~~~~~~~~~~~~
+Here is an example of how the hyperdatabase module would work in practice::
+>>> import hyperdb
+>>> db = hyperdb.Database("foo.db", "ping")
+>>> db
+<hyperdb.Database "foo.db" opened by "ping">
+>>> hyperdb.Class(db, "status", name=hyperdb.String())
+<hyperdb.Class "status">
+>>> _.setkey("name")
+>>> db.status.create(name="unread")
+1
+>>> db.status.create(name="in-progress")
+2
+>>> db.status.create(name="testing")
+3
+>>> db.status.create(name="resolved")
+4
+>>> db.status.count()
+4
+>>> db.status.list()
+[1, 2, 3, 4]
+>>> db.status.lookup("in-progress")
+2
+>>> db.status.retire(3)
+>>> db.status.list()
+[1, 2, 4]
+>>> hyperdb.Class(db, "issue", title=hyperdb.String(), status=hyperdb.Link("status"))
+<hyperdb.Class "issue">
+>>> db.issue.create(title="spam", status=1)
+1
+>>> db.issue.create(title="eggs", status=2)
+2
+>>> db.issue.create(title="ham", status=4)
+3
+>>> db.issue.create(title="arguments", status=2)
+4
+>>> db.issue.create(title="abuse", status=1)
+5
+>>> hyperdb.Class(db, "user", username=hyperdb.Key(), password=hyperdb.String())
+<hyperdb.Class "user">
+>>> db.issue.addprop(fixer=hyperdb.Link("user"))
+>>> db.issue.getprops()
+{"title": <hyperdb.String>, "status": <hyperdb.Link to "status">,
+"user": <hyperdb.Link to "user">}
+>>> db.issue.set(5, status=2)
+>>> db.issue.get(5, "status")
+2
+>>> db.status.get(2, "name")
+"in-progress"
+>>> db.issue.get(5, "title")
+"abuse"
+>>> db.issue.find("status", db.status.lookup("in-progress"))
+[2, 4, 5]
+>>> db.issue.history(5)
+[(<Date 2000-06-28.19:09:43>, "ping", "create", {"title": "abuse", "status": 1}),
+(<Date 2000-06-28.19:11:04>, "ping", "set", {"status": 2})]
+>>> db.status.history(1)
+[(<Date 2000-06-28.19:09:43>, "ping", "link", ("issue", 5, "status")),
+(<Date 2000-06-28.19:11:04>, "ping", "unlink", ("issue", 5, "status"))]
+>>> db.status.history(2)
+[(<Date 2000-06-28.19:11:04>, "ping", "link", ("issue", 5, "status"))]
+For the purposes of journalling, when a Multilink property is
+set to a new list of nodes, the hyperdatabase compares the old
+list to the new list.
+The journal records "unlink" events for all the nodes that appear
+in the old list but not the new list,
+and "link" events for
+all the nodes that appear in the new list but not in the old list.
+Roundup Database
+----------------
+The Roundup database layer is implemented on top of the
+hyperdatabase and mediates calls to the database.
+Some of the classes in the Roundup database are considered
+issue classes.
+The Roundup database layer adds detectors and user nodes,
+and on issues it provides mail spools, nosy lists, and superseders.
+TODO: where functionality is implemented.
+Reserved Classes
+~~~~~~~~~~~~~~~~
+Internal to this layer we reserve three special classes
+of nodes that are not issues.
+Users
+""""""""""""
+Users are stored in the hyperdatabase as nodes of
+class "user".  The "user" class has the definition::
+hyperdb.Class(db, "user", username=hyperdb.String(),
+password=hyperdb.String(),
+address=hyperdb.String())
+db.user.setkey("username")
+Messages
+"""""""""""""""
+E-mail messages are represented by hyperdatabase nodes of class "msg".
+The actual text content of the messages is stored in separate files.
+(There's no advantage to be gained by stuffing them into the
+hyperdatabase, and if messages are stored in ordinary text files,
+they can be grepped from the command line.)  The text of a message is
+saved in a file named after the message node designator (e.g. "msg23")
+for the sake of the command interface (see below).  Attachments are
+stored separately and associated with "file" nodes.
+The "msg" class has the definition::
+hyperdb.Class(db, "msg", author=hyperdb.Link("user"),
+recipients=hyperdb.Multilink("user"),
+date=hyperdb.Date(),
+summary=hyperdb.String(),
+files=hyperdb.Multilink("file"))
+The "author" property indicates the author of the message
+(a "user" node must exist in the hyperdatabase for any messages
+that are stored in the system).
+The "summary" property contains a summary of the message for display
+in a message index.
+Files
+""""""""""""
+Submitted files are represented by hyperdatabase
+nodes of class "file".  Like e-mail messages, the file content
+is stored in files outside the database,
+named after the file node designator (e.g. "file17").
+The "file" class has the definition::
+hyperdb.Class(db, "file", user=hyperdb.Link("user"),
+name=hyperdb.String(),
+type=hyperdb.String())
+The "user" property indicates the user who submitted the
+file, the "name" property holds the original name of the file,
+and the "type" property holds the MIME type of the file as received.
+Issue Classes
+~~~~~~~~~~~~~
+All issues have the following standard properties:
+=========== ==========================
+Property    Definition
+=========== ==========================
+title       hyperdb.String()
+messages    hyperdb.Multilink("msg")
+files       hyperdb.Multilink("file")
+nosy        hyperdb.Multilink("user")
+superseder  hyperdb.Multilink("issue")
+=========== ==========================
+Also, two Date properties named "creation" and "activity" are
+fabricated by the Roundup database layer.  By "fabricated" we
+mean that no such properties are actually stored in the
+hyperdatabase, but when properties on issues are requested, the
+"creation" and "activity" properties are made available.
+The value of the "creation" property is the date when an issue was
+created, and the value of the "activity" property is the
+date when any property on the issue was last edited (equivalently,
+these are the dates on the first and last records in the issue's journal).
+Roundupdb Interface Specification
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The interface to a Roundup database delegates most method
+calls to the hyperdatabase, except for the following
+changes and additional methods::
+class Database:
+def getuid(self):
+"""Return the id of the "user" node associated with the user
+that owns this connection to the hyperdatabase."""
+class Class:
+# Overridden methods:
+def create(self, **propvalues):
+def set(self, **propvalues):
+def retire(self, nodeid):
+"""These operations trigger detectors and can be vetoed.  Attempts
+to modify the "creation" or "activity" properties cause a KeyError.
+"""
+# New methods:
+def audit(self, event, detector):
+def react(self, event, detector):
+"""Register a detector (see below for more details)."""
+class IssueClass(Class):
+# Overridden methods:
+def __init__(self, db, classname, **properties):
+"""The newly-created class automatically includes the "messages",
+"files", "nosy", and "superseder" properties.  If the 'properties'
+dictionary attempts to specify any of these properties or a
+"creation" or "activity" property, a ValueError is raised."""
+def get(self, nodeid, propname):
+def getprops(self):
+"""In addition to the actual properties on the node, these
+methods provide the "creation" and "activity" properties."""
+# New methods:
+def addmessage(self, nodeid, summary, text):
+"""Add a message to an issue's mail spool.
+A new "msg" node is constructed using the current date, the
+user that owns the database connection as the author, and
+the specified summary text.  The "files" and "recipients"
+fields are left empty.  The given text is saved as the body
+of the message and the node is appended to the "messages"
+field of the specified issue.
+"""
+def sendmessage(self, nodeid, msgid):
+"""Send a message to the members of an issue's nosy list.
+The message is sent only to users on the nosy list who are not
+already on the "recipients" list for the message.  These users
+are then added to the message's "recipients" list.
+"""
+Default Schema
+~~~~~~~~~~~~~~
+The default schema included with Roundup turns it into a
+typical software bug tracker.  The database is set up like this::
+pri = Class(db, "priority", name=hyperdb.String(), order=hyperdb.String())
+pri.setkey("name")
+pri.create(name="critical", order="1")
+pri.create(name="urgent", order="2")
+pri.create(name="bug", order="3")
+pri.create(name="feature", order="4")
+pri.create(name="wish", order="5")
+stat = Class(db, "status", name=hyperdb.String(), order=hyperdb.String())
+stat.setkey("name")
+stat.create(name="unread", order="1")
+stat.create(name="deferred", order="2")
+stat.create(name="chatting", order="3")
+stat.create(name="need-eg", order="4")
+stat.create(name="in-progress", order="5")
+stat.create(name="testing", order="6")
+stat.create(name="done-cbb", order="7")
+stat.create(name="resolved", order="8")
+Class(db, "keyword", name=hyperdb.String())
+Class(db, "issue", fixer=hyperdb.Multilink("user"),
+topic=hyperdb.Multilink("keyword"),
+priority=hyperdb.Link("priority"),
+status=hyperdb.Link("status"))
+(The "order" property hasn't been explained yet.  It
+gets used by the Web user interface for sorting.)
+The above isn't as pretty-looking as the schema specification
+in the first-stage submission, but it could be made just as easy
+with the addition of a convenience function like Choice
+for setting up the "priority" and "status" classes::
+def Choice(name, *options):
+cl = Class(db, name, name=hyperdb.String(), order=hyperdb.String())
+for i in range(len(options)):
+cl.create(name=option[i], order=i)
+return hyperdb.Link(name)
+Detector Interface
+------------------
+Detectors are Python functions that are triggered on certain
+kinds of events.  The definitions of the
+functions live in Python modules placed in a directory set aside
+for this purpose.  Importing the Roundup database module also
+imports all the modules in this directory, and the ``init()``
+function of each module is called when a database is opened to
+provide it a chance to register its detectors.
+There are two kinds of detectors:
+1. an auditor is triggered just before modifying an node
+2. a reactor is triggered just after an node has been modified
+When the Roundup database is about to perform a
+``create()``, ``set()``, or ``retire()``
+operation, it first calls any *auditors* that
+have been registered for that operation on that class.
+Any auditor may raise a *Reject* exception
+to abort the operation.
+If none of the auditors raises an exception, the database
+proceeds to carry out the operation.  After it's done, it
+then calls all of the *reactors* that have been registered
+for the operation.
+Detector Interface Specification
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The ``audit()`` and ``react()`` methods
+register detectors on a given class of nodes::
+class Class:
+def audit(self, event, detector):
+"""Register an auditor on this class.
+'event' should be one of "create", "set", or "retire".
+'detector' should be a function accepting four arguments.
+"""
+def react(self, event, detector):
+"""Register a reactor on this class.
+'event' should be one of "create", "set", or "retire".
+'detector' should be a function accepting four arguments.
+"""
+Auditors are called with the arguments::
+audit(db, cl, nodeid, newdata)
+where ``db`` is the database, ``cl`` is an
+instance of Class or IssueClass within the database, and ``newdata``
+is a dictionary mapping property names to values.
+For a ``create()``
+operation, the ``nodeid`` argument is None and newdata
+contains all of the initial property values with which the node
+is about to be created.
+For a ``set()`` operation, newdata
+contains only the names and values of properties that are about
+to be changed.
+For a ``retire()`` operation, newdata is None.
+Reactors are called with the arguments::
+react(db, cl, nodeid, olddata)
+where ``db`` is the database, ``cl`` is an
+instance of Class or IssueClass within the database, and ``olddata``
+is a dictionary mapping property names to values.
+For a ``create()``
+operation, the ``nodeid`` argument is the id of the
+newly-created node and ``olddata`` is None.
+For a ``set()`` operation, ``olddata``
+contains the names and previous values of properties that were changed.
+For a ``retire()`` operation, ``nodeid`` is the
+id of the retired node and ``olddata`` is None.
+Detector Example
+~~~~~~~~~~~~~~~~
+Here is an example of detectors written for a hypothetical
+project-management application, where users can signal approval
+of a project by adding themselves to an "approvals" list, and
+a project proceeds when it has three approvals::
+# Permit users only to add themselves to the "approvals" list.
+def check_approvals(db, cl, id, newdata):
+if newdata.has_key("approvals"):
+if cl.get(id, "status") == db.status.lookup("approved"):
+raise Reject, "You can't modify the approvals list " \
+"for a project that has already been approved."
+old = cl.get(id, "approvals")
+new = newdata["approvals"]
+for uid in old:
+if uid not in new and uid != db.getuid():
+raise Reject, "You can't remove other users from the "
+"approvals list; you can only remove yourself."
+for uid in new:
+if uid not in old and uid != db.getuid():
+raise Reject, "You can't add other users to the approvals "
+"list; you can only add yourself."
+# When three people have approved a project, change its
+# status from "pending" to "approved".
+def approve_project(db, cl, id, olddata):
+if olddata.has_key("approvals") and len(cl.get(id, "approvals")) == 3:
+if cl.get(id, "status") == db.status.lookup("pending"):
+cl.set(id, status=db.status.lookup("approved"))
+def init(db):
+db.project.audit("set", check_approval)
+db.project.react("set", approve_project)
+Here is another example of a detector that can allow or prevent
+the creation of new nodes.  In this scenario, patches for a software
+project are submitted by sending in e-mail with an attached file,
+and we want to ensure that there are text/plain attachments on
+the message.  The maintainer of the package can then apply the
+patch by setting its status to "applied"::
+# Only accept attempts to create new patches that come with patch files.
+def check_new_patch(db, cl, id, newdata):
+if not newdata["files"]:
+raise Reject, "You can't submit a new patch without " \
+"attaching a patch file."
+for fileid in newdata["files"]:
+if db.file.get(fileid, "type") != "text/plain":
+raise Reject, "Submitted patch files must be text/plain."
+# When the status is changed from "approved" to "applied", apply the patch.
+def apply_patch(db, cl, id, olddata):
+if cl.get(id, "status") == db.status.lookup("applied") and \
+olddata["status"] == db.status.lookup("approved"):
+# ...apply the patch...
+def init(db):
+db.patch.audit("create", check_new_patch)
+db.patch.react("set", apply_patch)
+Command Interface
+-----------------
+The command interface is a very simple and minimal interface,
+intended only for quick searches and checks from the shell prompt.
+(Anything more interesting can simply be written in Python using
+the Roundup database module.)
+Command Interface Specification
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+A single command, roundup, provides basic access to
+the hyperdatabase from the command line::
+roundup get [-list] designator[, designator,...] propname
+roundup set designator[, designator,...] propname=value ...
+roundup find [-list] classname propname=value ...
+TODO: more stuff here
+Property values are represented as strings in command arguments
+and in the printed results:
+- Strings are, well, strings.
+- Numbers are displayed the same as strings.
+- Booleans are displayed as 'Yes' or 'No'.
+- Date values are printed in the full date format in the local
+time zone, and accepted in the full format or any of the partial
+formats explained above.
+- Link values are printed as node designators.  When given as
+an argument, node designators and key strings are both accepted.
+- Multilink values are printed as lists of node designators
+joined by commas.  When given as an argument, node designators
+and key strings are both accepted; an empty string, a single node,
+or a list of nodes joined by commas is accepted.
+When multiple nodes are specified to the
+roundup get or roundup set
+commands, the specified properties are retrieved or set
+on all the listed nodes.
+When multiple results are returned by the roundup get
+or roundup find commands, they are printed one per
+line (default) or joined by commas (with the -list) option.
+Usage Example
+~~~~~~~~~~~~~
+To find all messages regarding in-progress issues that
+contain the word "spam", for example, you could execute the
+following command from the directory where the database
+dumps its files::
+shell% for issue in `roundup find issue status=in-progress`; do
+> grep -l spam `roundup get $issue messages`
+> done
+msg23
+msg49
+msg50
+msg61
+shell%
+Or, using the -list option, this can be written as a single command::
+shell% grep -l spam `roundup get \
+\`roundup find -list issue status=in-progress\` messages`
+msg23
+msg49
+msg50
+msg61
+shell%
+E-mail User Interface
+---------------------
+The Roundup system must be assigned an e-mail address
+at which to receive mail.  Messages should be piped to
+the Roundup mail-handling script by the mail delivery
+system (e.g. using an alias beginning with "|" for sendmail).
+Message Processing
+~~~~~~~~~~~~~~~~~~
+Incoming messages are examined for multiple parts.
+In a multipart/mixed message or part, each subpart is
+extracted and examined.  In a multipart/alternative
+message or part, we look for a text/plain subpart and
+ignore the other parts.  The text/plain subparts are
+assembled to form the textual body of the message, to
+be stored in the file associated with a "msg" class node.
+Any parts of other types are each stored in separate
+files and given "file" class nodes that are linked to
+the "msg" node.
+The "summary" property on message nodes is taken from
+the first non-quoting section in the message body.
+The message body is divided into sections by blank lines.
+Sections where the second and all subsequent lines begin
+with a ">" or "|" character are considered "quoting
+sections".  The first line of the first non-quoting
+section becomes the summary of the message.
+All of the addresses in the To: and Cc: headers of the
+incoming message are looked up among the user nodes, and
+the corresponding users are placed in the "recipients"
+property on the new "msg" node.  The address in the From:
+header similarly determines the "author" property of the
+new "msg" node.
+The default handling for
+addresses that don't have corresponding users is to create
+new users with no passwords and a username equal to the
+address.  (The web interface does not permit logins for
+users with no passwords.)  If we prefer to reject mail from
+outside sources, we can simply register an auditor on the
+"user" class that prevents the creation of user nodes with
+no passwords.
+The subject line of the incoming message is examined to
+determine whether the message is an attempt to create a new
+issue or to discuss an existing issue.  A designator enclosed
+in square brackets is sought as the first thing on the
+subject line (after skipping any "Fwd:" or "Re:" prefixes).
+If an issue designator (class name and id number) is found
+there, the newly created "msg" node is added to the "messages"
+property for that issue, and any new "file" nodes are added to
+the "files" property for the issue.
+If just an issue class name is found there, we attempt to
+create a new issue of that class with its "messages" property
+initialized to contain the new "msg" node and its "files"
+property initialized to contain any new "file" nodes.
+Both cases may trigger detectors (in the first case we
+are calling the set() method to add the message to the
+issue's spool; in the second case we are calling the
+create() method to create a new node).  If an auditor
+raises an exception, the original message is bounced back to
+the sender with the explanatory message given in the exception.
+Nosy Lists
+~~~~~~~~~~
+A standard detector is provided that watches for additions
+to the "messages" property.  When a new message is added, the
+detector sends it to all the users on the "nosy" list for the
+issue that are not already on the "recipients" list of the
+message.  Those users are then appended to the "recipients"
+property on the message, so multiple copies of a message
+are never sent to the same user.  The journal recorded by
+the hyperdatabase on the "recipients" property then provides
+a log of when the message was sent to whom.
+Setting Properties
+~~~~~~~~~~~~~~~~~~
+The e-mail interface also provides a simple way to set
+properties on issues.  At the end of the subject line,
+``propname=value`` pairs can be
+specified in square brackets, using the same conventions
+as for the roundup ``set`` shell command.
+Web User Interface
+------------------
+The web interface is provided by a CGI script that can be
+run under any web server.  A simple web server can easily be
+built on the standard CGIHTTPServer module, and
+should also be included in the distribution for quick
+out-of-the-box deployment.
+The user interface is constructed from a number of template
+files containing mostly HTML.  Among the HTML tags in templates
+are interspersed some nonstandard tags, which we use as
+placeholders to be replaced by properties and their values.
+Views and View Specifiers
+~~~~~~~~~~~~~~~~~~~~~~~~~
+There are two main kinds of views: *index* views and *issue* views.
+An index view displays a list of issues of a particular class,
+optionally sorted and filtered as requested.  An issue view
+presents the properties of a particular issue for editing
+and displays the message spool for the issue.
+A view specifier is a string that specifies
+all the options needed to construct a particular view.
+It goes after the URL to the Roundup CGI script or the
+web server to form the complete URL to a view.  When the
+result of selecting a link or submitting a form takes
+the user to a new view, the Web browser should be redirected
+to a canonical location containing a complete view specifier
+so that the view can be bookmarked.
+Displaying Properties
+~~~~~~~~~~~~~~~~~~~~~
+Properties appear in the user interface in three contexts:
+in indices, in editors, and as filters.  For each type of
+property, there are several display possibilities.  For example,
+in an index view, a string property may just be printed as
+a plain string, but in an editor view, that property should
+be displayed in an editable field.
+The display of a property is handled by functions in
+a displayers module.  Each function accepts at
+least three standard arguments -- the database, class name,
+and node id -- and returns a chunk of HTML.
+Displayer functions are triggered by <display>
+tags in templates.  The call attribute of the tag
+provides a Python expression for calling the displayer
+function.  The three standard arguments are inserted in
+front of the arguments given.  For example, the occurrence of::
+<display call="plain('status', max=30)">
+in a template triggers a call to::
+plain(db, "issue", 13, "status", max=30)
+when displaying issue 13 in the "issue" class.  The displayer
+functions can accept extra arguments to further specify
+details about the widgets that should be generated.  By defining new
+displayer functions, the user interface can be highly customized.
+Some of the standard displayer functions include:
+========= ====================================================================
+Function  Description
+========= ====================================================================
+plain     display a String property directly;
+display a Date property in a specified time zone with an option
+to omit the time from the date stamp; for a Link or Multilink
+property, display the key strings of the linked nodes (or the
+ids if the linked class has no key property)
+field     display a property like the
+plain displayer above, but in a text field
+to be edited
+menu      for a Link property, display
+a menu of the available choices
+link      for a Link or Multilink property,
+display the names of the linked nodes, hyperlinked to the
+issue views on those nodes
+count     for a Multilink property, display
+a count of the number of links in the list
+reldate   display a Date property in terms
+of an interval relative to the current date (e.g. "+ 3w", "- 2d").
+download  show a Link("file") or Multilink("file")
+property using links that allow you to download files
+checklist for a Link or Multilink property,
+display checkboxes for the available choices to permit filtering
+========= ====================================================================
+Index Views
+~~~~~~~~~~~
+An index view contains two sections: a filter section
+and an index section.
+The filter section provides some widgets for selecting
+which issues appear in the index.  The index section is
+a table of issues.
+Index View Specifiers
+"""""""""""""""""""""
+An index view specifier looks like this (whitespace
+has been added for clarity)::
+/issue?status=unread,in-progress,resolved&amp;
+topic=security,ui&amp;
+:group=+priority&amp;
+:sort=-activity&amp;
+:filters=status,topic&amp;
+:columns=title,status,fixer
+The index view is determined by two parts of the
+specifier: the layout part and the filter part.
+The layout part consists of the query parameters that
+begin with colons, and it determines the way that the
+properties of selected nodes are displayed.
+The filter part consists of all the other query parameters,
+and it determines the criteria by which nodes
+are selected for display.
+The filter part is interactively manipulated with
+the form widgets displayed in the filter section.  The
+layout part is interactively manipulated by clicking
+on the column headings in the table.
+The filter part selects the union of the
+sets of issues with values matching any specified Link
+properties and the intersection of the sets
+of issues with values matching any specified Multilink
+properties.
+The example specifies an index of "issue" nodes.
+Only issues with a "status" of either
+"unread" or "in-progres" or "resolved" are displayed,
+and only issues with "topic" values including both
+"security" and "ui" are displayed.  The issues
+are grouped by priority, arranged in ascending order;
+and within groups, sorted by activity, arranged in
+descending order.  The filter section shows filters
+for the "status" and "topic" properties, and the
+table includes columns for the "title", "status", and
+"fixer" properties.
+Associated with each issue class is a default
+layout specifier.  The layout specifier in the above
+example is the default layout to be provided with
+the default bug-tracker schema described above in
+section 4.4.
+Filter Section
+""""""""""""""
+The template for a filter section provides the
+filtering widgets at the top of the index view.
+Fragments enclosed in ``<property>...</property>``
+tags are included or omitted depending on whether the
+view specifier requests a filter for a particular property.
+Here's a simple example of a filter template::
+<property name=status>
+<display call="checklist('status')">
+</property>
+<br>
+<property name=priority>
+<display call="checklist('priority')">
+</property>
+<br>
+<property name=fixer>
+<display call="menu('fixer')">
+</property>
+Index Section
+"""""""""""""
+The template for an index section describes one row of
+the index table.
+Fragments enclosed in ``<property>...</property>``
+tags are included or omitted depending on whether the
+view specifier requests a column for a particular property.
+The table cells should contain <display> tags
+to display the values of the issue's properties.
+Here's a simple example of an index template::
+<tr>
+<property name=title>
+<td><display call="plain('title', max=50)"></td>
+</property>
+<property name=status>
+<td><display call="plain('status')"></td>
+</property>
+<property name=fixer>
+<td><display call="plain('fixer')"></td>
+</property>
+</tr>
+Sorting
+""""""""""""""
+String and Date values are sorted in the natural way.
+Link properties are sorted according to the value of the
+"order" property on the linked nodes if it is present; or
+otherwise on the key string of the linked nodes; or
+finally on the node ids.  Multilink properties are
+sorted according to how many links are present.
+Issue Views
+~~~~~~~~~~~
+An issue view contains an editor section and a spool section.
+At the top of an issue view, links to superseding and superseded
+issues are always displayed.
+Issue View Specifiers
+"""""""""""""""""""""
+An issue view specifier is simply the issue's designator::
+/patch23
+Editor Section
+""""""""""""""
+The editor section is generated from a template
+containing <display> tags to insert
+the appropriate widgets for editing properties.
+Here's an example of a basic editor template::
+<table>
+<tr>
+<td colspan=2>
+<display call="field('title', size=60)">
+</td>
+</tr>
+<tr>
+<td>
+<display call="field('fixer', size=30)">
+</td>
+<td>
+<display call="menu('status')>
+</td>
+</tr>
+<tr>
+<td>
+<display call="field('nosy', size=30)">
+</td>
+<td>
+<display call="menu('priority')>
+</td>
+</tr>
+<tr>
+<td colspan=2>
+<display call="note()">
+</td>
+</tr>
+</table>
+As shown in the example, the editor template can also
+request the display of a "note" field, which is a
+text area for entering a note to go along with a change.
+When a change is submitted, the system automatically
+generates a message describing the changed properties.
+The message displays all of the property values on the
+issue and indicates which ones have changed.
+An example of such a message might be this::
+title: Polly Parrot is dead
+priority: critical
+status: unread -> in-progress
+fixer: (none)
+keywords: parrot,plumage,perch,nailed,dead
+If a note is given in the "note" field, the note is
+appended to the description.  The message is then added
+to the issue's message spool (thus triggering the standard
+detector to react by sending out this message to the nosy list).
+Spool Section
+"""""""""""""
+The spool section lists messages in the issue's "messages"
+property.  The index of messages displays the "date", "author",
+and "summary" properties on the message nodes, and selecting a
+message takes you to its content.
+Deployment Scenarios
+--------------------
+The design described above should be general enough
+to permit the use of Roundup for bug tracking, managing
+projects, managing patches, or holding discussions.  By
+using nodes of multiple types, one could deploy a system
+that maintains requirement specifications, catalogs bugs,
+and manages submitted patches, where patches could be
+linked to the bugs and requirements they address.
+Acknowledgements
+----------------
+My thanks are due to Christy Heyl for
+reviewing and contributing suggestions to this paper
+and motivating me to get it done, and to
+Jesse Vincent, Mark Miller, Christopher Simons,
+Jeff Dunmall, Wayne Gramlich, and Dean Tribble for
+their assistance with the first-round submission.
+Changes to this document
+------------------------
+- Added Boolean and Number types
+- Added section Hyperdatabase Implementations
+- "Item" has been renamed to "Issue" to account for the more specific nature
+of the Class.

Mercurial > p > roundup > code

comparison doc/design.txt @ 907:38a74d1351c5