--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/doc/design.txt	Mon Jul 29 00:54:41 2002 +0000
@@ -0,0 +1,1399 @@
+========================================================
+Roundup - An Issue-Tracking System for Knowledge Workers
+========================================================
+
+:Authors: Ka-Ping Yee (original__), Richard Jones (implementation)
+
+__ spec.html
+
+.. contents::
+
+Introduction
+---------------
+
+This document presents a description of the components
+of the Roundup system and specifies their interfaces and
+behaviour in sufficient detail to guide an implementation.
+For the philosophy and rationale behind the Roundup design,
+see the first-round Software Carpentry submission for Roundup.
+This document fleshes out that design as well as specifying
+interfaces so that the components can be developed separately.
+
+
+The Layer Cake
+-----------------
+
+Lots of software design documents come with a picture of
+a cake.  Everybody seems to like them.  I also like cakes
+(i think they are tasty).  So i, too, shall include
+a picture of a cake here::
+
+     _________________________________________________________________________
+    |  E-mail Client   |   Web Browser   |   Detector Scripts   |    Shell    |
+    |------------------+-----------------+----------------------+-------------|
+    |   E-mail User    |    Web User     |      Detector        |   Command   | 
+    |-------------------------------------------------------------------------|
+    |                         Roundup Database Layer                          |
+    |-------------------------------------------------------------------------|
+    |                          Hyperdatabase Layer                            |
+    |-------------------------------------------------------------------------|
+    |                             Storage Layer                               |
+     -------------------------------------------------------------------------
+
+The colourful parts of the cake are part of our system;
+the faint grey parts of the cake are external components.
+
+I will now proceed to forgo all table manners and
+eat from the bottom of the cake to the top.  You may want
+to stand back a bit so you don't get covered in crumbs.
+
+
+Hyperdatabase
+-------------
+
+The lowest-level component to be implemented is the hyperdatabase.
+The hyperdatabase is intended to be
+a flexible data store that can hold configurable data in
+records which we call nodes.
+
+The hyperdatabase is implemented on top of the storage layer,
+an external module for storing its data.  The storage layer could
+be a third-party RDBMS; for a "batteries-included" distribution,
+implementing the hyperdatabase on the standard bsddb
+module is suggested.
+
+Dates and Date Arithmetic
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Before we get into the hyperdatabase itself, we need a
+way of handling dates.  The hyperdatabase module provides
+Timestamp objects for
+representing date-and-time stamps and Interval objects for
+representing date-and-time intervals.
+
+As strings, date-and-time stamps are specified with
+the date in international standard format
+(``yyyy-mm-dd``)
+joined to the time (``hh:mm:ss``)
+by a period "``.``".  Dates in
+this form can be easily compared and are fairly readable
+when printed.  An example of a valid stamp is
+"``2000-06-24.13:03:59``".
+We'll call this the "full date format".  When Timestamp objects are
+printed as strings, they appear in the full date format with
+the time always given in GMT.  The full date format is always
+exactly 19 characters long.
+
+For user input, some partial forms are also permitted:
+the whole time or just the seconds may be omitted; and the whole date
+may be omitted or just the year may be omitted.  If the time is given,
+the time is interpreted in the user's local time zone.
+The Date constructor takes care of these conversions.
+In the following examples, suppose that ``yyyy`` is the current year,
+``mm`` is the current month, and ``dd`` is the current
+day of the month; and suppose that the user is on Eastern Standard Time.
+
+-   "2000-04-17" means <Date 2000-04-17.00:00:00>
+-   "01-25" means <Date yyyy-01-25.00:00:00>
+-   "2000-04-17.03:45" means <Date 2000-04-17.08:45:00>
+-   "08-13.22:13" means <Date yyyy-08-14.03:13:00>
+-   "11-07.09:32:43" means <Date yyyy-11-07.14:32:43>
+-   "14:25" means
+-   <Date yyyy-mm-dd.19:25:00>
+-   "8:47:11" means
+-   <Date yyyy-mm-dd.13:47:11>
+-   the special date "." means "right now"
+
+
+Date intervals are specified using the suffixes
+"y", "m", and "d".  The suffix "w" (for "week") means 7 days.
+Time intervals are specified in hh:mm:ss format (the seconds
+may be omitted, but the hours and minutes may not).
+
+-   "3y" means three years
+-   "2y 1m" means two years and one month
+-   "1m 25d" means one month and 25 days
+-   "2w 3d" means two weeks and three days
+-   "1d 2:50" means one day, two hours, and 50 minutes
+-   "14:00" means 14 hours
+-   "0:04:33" means four minutes and 33 seconds
+
+
+The Date class should understand simple date expressions of the form 
+*stamp* ``+`` *interval* and *stamp* ``-`` *interval*.
+When adding or subtracting intervals involving months or years, the
+components are handled separately.  For example, when evaluating
+"``2000-06-25 + 1m 10d``", we first add one month to
+get 2000-07-25, then add 10 days to get
+2000-08-04 (rather than trying to decide whether
+1m 10d means 38 or 40 or 41 days).
+
+Here is an outline of the Date and Interval classes::
+
+    class Date:
+        def __init__(self, spec, offset):
+            """Construct a date given a specification and a time zone offset.
+
+            'spec' is a full date or a partial form, with an optional
+            added or subtracted interval.  'offset' is the local time
+            zone offset from GMT in hours.
+            """
+
+        def __add__(self, interval):
+            """Add an interval to this date to produce another date."""
+
+        def __sub__(self, interval):
+            """Subtract an interval from this date to produce another date."""
+
+        def __cmp__(self, other):
+            """Compare this date to another date."""
+
+        def __str__(self):
+            """Return this date as a string in the yyyy-mm-dd.hh:mm:ss format."""
+
+        def local(self, offset):
+            """Return this date as yyyy-mm-dd.hh:mm:ss in a local time zone."""
+
+    class Interval:
+        def __init__(self, spec):
+            """Construct an interval given a specification."""
+
+        def __cmp__(self, other):
+            """Compare this interval to another interval."""
+            
+        def __str__(self):
+            """Return this interval as a string."""
+
+
+
+Here are some examples of how these classes would behave in practice.
+For the following examples, assume that we are on Eastern Standard
+Time and the current local time is 19:34:02 on 25 June 2000::
+
+    >>> Date(".")
+    <Date 2000-06-26.00:34:02>
+    >>> _.local(-5)
+    "2000-06-25.19:34:02"
+    >>> Date(". + 2d")
+    <Date 2000-06-28.00:34:02>
+    >>> Date("1997-04-17", -5)
+    <Date 1997-04-17.00:00:00>
+    >>> Date("01-25", -5)
+    <Date 2000-01-25.00:00:00>
+    >>> Date("08-13.22:13", -5)
+    <Date 2000-08-14.03:13:00>
+    >>> Date("14:25", -5)
+    <Date 2000-06-25.19:25:00>
+    >>> Interval("  3w  1  d  2:00")
+    <Interval 22d 2:00>
+    >>> Date(". + 2d") - Interval("3w")
+    <Date 2000-06-07.00:34:02>
+
+Nodes and Classes
+~~~~~~~~~~~~~~~~~
+
+Nodes contain data in properties.  To Python, these
+properties are presented as the key-value pairs of a dictionary.
+Each node belongs to a class which defines the names
+and types of its properties.  The database permits the creation
+and modification of classes as well as nodes.
+
+Identifiers and Designators
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Each node has a numeric identifier which is unique among
+nodes in its class.  The nodes are numbered sequentially
+within each class in order of creation, starting from 1.
+The designator
+for a node is a way to identify a node in the database, and
+consists of the name of the node's class concatenated with
+the node's numeric identifier.
+
+For example, if "spam" and "eggs" are classes, the first
+node created in class "spam" has id 1 and designator "spam1".
+The first node created in class "eggs" also has id 1 but has
+the distinct designator "eggs1".  Node designators are
+conventionally enclosed in square brackets when mentioned
+in plain text.  This permits a casual mention of, say,
+"[patch37]" in an e-mail message to be turned into an active
+hyperlink.
+
+Property Names and Types
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+Property names must begin with a letter.
+
+A property may be one of five basic types:
+
+- String properties are for storing arbitrary-length strings.
+
+- Boolean properties are for storing true/false, or yes/no values.
+
+- Number properties are for storing numeric values.
+
+- Date properties store date-and-time stamps.
+  Their values are Timestamp objects.
+
+- A Link property refers to a single other node
+  selected from a specified class.  The class is part of the property;
+  the value is an integer, the id of the chosen node.
+
+- A Multilink property refers to possibly many nodes
+  in a specified class.  The value is a list of integers.
+
+*None* is also a permitted value for any of these property
+types.  An attempt to store None into a Multilink property stores an empty list.
+
+A property that is not specified will return as None from a *get*
+operation.
+
+Hyperdb Interface Specification
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The hyperdb module provides property objects to designate
+the different kinds of properties.  These objects are used when
+specifying what properties belong in classes::
+
+    class String:
+        def __init__(self, indexme='no'):
+            """An object designating a String property."""
+
+    class Boolean:
+        def __init__(self):
+            """An object designating a Boolean property."""
+
+    class Number:
+        def __init__(self):
+            """An object designating a Number property."""
+
+    class Date:
+        def __init__(self):
+            """An object designating a Date property."""
+
+    class Link:
+        def __init__(self, classname, do_journal='yes'):
+            """An object designating a Link property that links to
+            nodes in a specified class.
+
+            If the do_journal argument is not 'yes' then changes to
+            the property are not journalled in the linked node.
+            """
+
+    class Multilink:
+        def __init__(self, classname, do_journal='yes'):
+            """An object designating a Multilink property that links
+            to nodes in a specified class.
+
+            If the do_journal argument is not 'yes' then changes to
+            the property are not journalled in the linked node(s).
+            """
+
+
+Here is the interface provided by the hyperdatabase::
+
+    class Database:
+        """A database for storing records containing flexible data types."""
+
+        def __init__(self, storagelocator, journaltag):
+            """Open a hyperdatabase given a specifier to some storage.
+
+            The meaning of 'storagelocator' depends on the particular
+            implementation of the hyperdatabase.  It could be a file name,
+            a directory path, a socket descriptor for a connection to a
+            database over the network, etc.
+
+            The 'journaltag' is a token that will be attached to the journal
+            entries for any edits done on the database.  If 'journaltag' is
+            None, the database is opened in read-only mode: the Class.create(),
+            Class.set(), and Class.retire() methods are disabled.
+            """
+
+        def __getattr__(self, classname):
+            """A convenient way of calling self.getclass(classname)."""
+
+        def getclasses(self):
+            """Return a list of the names of all existing classes."""
+
+        def getclass(self, classname):
+            """Get the Class object representing a particular class.
+
+            If 'classname' is not a valid class name, a KeyError is raised.
+            """
+
+    class Class:
+        """The handle to a particular class of nodes in a hyperdatabase."""
+
+        def __init__(self, db, classname, **properties):
+            """Create a new class with a given name and property specification.
+
+            'classname' must not collide with the name of an existing class,
+            or a ValueError is raised.  The keyword arguments in 'properties'
+            must map names to property objects, or a TypeError is raised.
+            """
+
+        # Editing nodes:
+
+        def create(self, **propvalues):
+            """Create a new node of this class and return its id.
+
+            The keyword arguments in 'propvalues' map property names to values.
+            The values of arguments must be acceptable for the types of their
+            corresponding properties or a TypeError is raised.  If this class
+            has a key property, it must be present and its value must not
+            collide with other key strings or a ValueError is raised.  Any other
+            properties on this class that are missing from the 'propvalues'
+            dictionary are set to None.  If an id in a link or multilink
+            property does not refer to a valid node, an IndexError is raised.
+            """
+
+        def get(self, nodeid, propname):
+            """Get the value of a property on an existing node of this class.
+
+            'nodeid' must be the id of an existing node of this class or an
+            IndexError is raised.  'propname' must be the name of a property
+            of this class or a KeyError is raised.
+            """
+
+        def set(self, nodeid, **propvalues):
+            """Modify a property on an existing node of this class.
+            
+            'nodeid' must be the id of an existing node of this class or an
+            IndexError is raised.  Each key in 'propvalues' must be the name
+            of a property of this class or a KeyError is raised.  All values
+            in 'propvalues' must be acceptable types for their corresponding
+            properties or a TypeError is raised.  If the value of the key
+            property is set, it must not collide with other key strings or a
+            ValueError is raised.  If the value of a Link or Multilink
+            property contains an invalid node id, a ValueError is raised.
+            """
+
+        def retire(self, nodeid):
+            """Retire a node.
+            
+            The properties on the node remain available from the get() method,
+            and the node's id is never reused.  Retired nodes are not returned
+            by the find(), list(), or lookup() methods, and other nodes may
+            reuse the values of their key properties.
+            """
+
+        def history(self, nodeid):
+            """Retrieve the journal of edits on a particular node.
+
+            'nodeid' must be the id of an existing node of this class or an
+            IndexError is raised.
+
+            The returned list contains tuples of the form
+
+                (date, tag, action, params)
+
+            'date' is a Timestamp object specifying the time of the change and
+            'tag' is the journaltag specified when the database was opened.
+            'action' may be:
+
+                'create' or 'set' -- 'params' is a dictionary of property values
+                'link' or 'unlink' -- 'params' is (classname, nodeid, propname)
+                'retire' -- 'params' is None
+            """
+
+        # Locating nodes:
+
+        def setkey(self, propname):
+            """Select a String property of this class to be the key property.
+
+            'propname' must be the name of a String property of this class or
+            None, or a TypeError is raised.  The values of the key property on
+            all existing nodes must be unique or a ValueError is raised.
+            """
+
+        def getkey(self):
+            """Return the name of the key property for this class or None."""
+
+        def lookup(self, keyvalue):
+            """Locate a particular node by its key property and return its id.
+
+            If this class has no key property, a TypeError is raised.  If the
+            'keyvalue' matches one of the values for the key property among
+            the nodes in this class, the matching node's id is returned;
+            otherwise a KeyError is raised.
+            """
+
+        def find(self, propname, nodeid):
+            """Get the ids of nodes in this class which link to a given node.
+            
+            'propname' must be the name of a property in this class, or a
+            KeyError is raised.  That property must be a Link or Multilink
+            property, or a TypeError is raised.  'nodeid' must be the id of
+            an existing node in the class linked to by the given property,
+            or an IndexError is raised.
+            """
+
+        def list(self):
+            """Return a list of the ids of the active nodes in this class."""
+
+        def count(self):
+            """Get the number of nodes in this class.
+
+            If the returned integer is 'numnodes', the ids of all the nodes
+            in this class run from 1 to numnodes, and numnodes+1 will be the
+            id of the next node to be created in this class.
+            """
+
+        # Manipulating properties:
+
+        def getprops(self):
+            """Return a dictionary mapping property names to property objects."""
+
+        def addprop(self, **properties):
+            """Add properties to this class.
+
+            The keyword arguments in 'properties' must map names to property
+            objects, or a TypeError is raised.  None of the keys in 'properties'
+            may collide with the names of existing properties, or a ValueError
+            is raised before any properties have been added.
+            """
+
+TODO: additional methods
+
+Hyperdatabase Implementations
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Hyperdatabase implementations exist to create the interface described in the
+`hyperdb interface specification`_
+over an existing storage mechanism. Examples are relational databases,
+\*dbm key-value databases, and so on.
+
+TODO: finish
+
+
+Application Example
+~~~~~~~~~~~~~~~~~~~
+
+Here is an example of how the hyperdatabase module would work in practice::
+
+    >>> import hyperdb
+    >>> db = hyperdb.Database("foo.db", "ping")
+    >>> db
+    <hyperdb.Database "foo.db" opened by "ping">
+    >>> hyperdb.Class(db, "status", name=hyperdb.String())
+    <hyperdb.Class "status">
+    >>> _.setkey("name")
+    >>> db.status.create(name="unread")
+    1
+    >>> db.status.create(name="in-progress")
+    2
+    >>> db.status.create(name="testing")
+    3
+    >>> db.status.create(name="resolved")
+    4
+    >>> db.status.count()
+    4
+    >>> db.status.list()
+    [1, 2, 3, 4]
+    >>> db.status.lookup("in-progress")
+    2
+    >>> db.status.retire(3)
+    >>> db.status.list()
+    [1, 2, 4]
+    >>> hyperdb.Class(db, "issue", title=hyperdb.String(), status=hyperdb.Link("status"))
+    <hyperdb.Class "issue">
+    >>> db.issue.create(title="spam", status=1)
+    1
+    >>> db.issue.create(title="eggs", status=2)
+    2
+    >>> db.issue.create(title="ham", status=4)
+    3
+    >>> db.issue.create(title="arguments", status=2)
+    4
+    >>> db.issue.create(title="abuse", status=1)
+    5
+    >>> hyperdb.Class(db, "user", username=hyperdb.Key(), password=hyperdb.String())
+    <hyperdb.Class "user">
+    >>> db.issue.addprop(fixer=hyperdb.Link("user"))
+    >>> db.issue.getprops()
+    {"title": <hyperdb.String>, "status": <hyperdb.Link to "status">,
+     "user": <hyperdb.Link to "user">}
+    >>> db.issue.set(5, status=2)
+    >>> db.issue.get(5, "status")
+    2
+    >>> db.status.get(2, "name")
+    "in-progress"
+    >>> db.issue.get(5, "title")
+    "abuse"
+    >>> db.issue.find("status", db.status.lookup("in-progress"))
+    [2, 4, 5]
+    >>> db.issue.history(5)
+    [(<Date 2000-06-28.19:09:43>, "ping", "create", {"title": "abuse", "status": 1}),
+     (<Date 2000-06-28.19:11:04>, "ping", "set", {"status": 2})]
+    >>> db.status.history(1)
+    [(<Date 2000-06-28.19:09:43>, "ping", "link", ("issue", 5, "status")),
+     (<Date 2000-06-28.19:11:04>, "ping", "unlink", ("issue", 5, "status"))]
+    >>> db.status.history(2)
+    [(<Date 2000-06-28.19:11:04>, "ping", "link", ("issue", 5, "status"))]
+
+
+For the purposes of journalling, when a Multilink property is
+set to a new list of nodes, the hyperdatabase compares the old
+list to the new list.
+The journal records "unlink" events for all the nodes that appear
+in the old list but not the new list,
+and "link" events for
+all the nodes that appear in the new list but not in the old list.
+
+
+Roundup Database
+----------------
+
+The Roundup database layer is implemented on top of the
+hyperdatabase and mediates calls to the database.
+Some of the classes in the Roundup database are considered
+issue classes.
+The Roundup database layer adds detectors and user nodes,
+and on issues it provides mail spools, nosy lists, and superseders.
+
+TODO: where functionality is implemented.
+
+Reserved Classes
+~~~~~~~~~~~~~~~~
+
+Internal to this layer we reserve three special classes
+of nodes that are not issues.
+
+Users
+""""""""""""
+
+Users are stored in the hyperdatabase as nodes of
+class "user".  The "user" class has the definition::
+
+    hyperdb.Class(db, "user", username=hyperdb.String(),
+                              password=hyperdb.String(),
+                              address=hyperdb.String())
+    db.user.setkey("username")
+
+Messages
+"""""""""""""""
+
+E-mail messages are represented by hyperdatabase nodes of class "msg".
+The actual text content of the messages is stored in separate files.
+(There's no advantage to be gained by stuffing them into the
+hyperdatabase, and if messages are stored in ordinary text files,
+they can be grepped from the command line.)  The text of a message is
+saved in a file named after the message node designator (e.g. "msg23")
+for the sake of the command interface (see below).  Attachments are
+stored separately and associated with "file" nodes.
+The "msg" class has the definition::
+
+    hyperdb.Class(db, "msg", author=hyperdb.Link("user"),
+                             recipients=hyperdb.Multilink("user"),
+                             date=hyperdb.Date(),
+                             summary=hyperdb.String(),
+                             files=hyperdb.Multilink("file"))
+
+The "author" property indicates the author of the message
+(a "user" node must exist in the hyperdatabase for any messages
+that are stored in the system).
+The "summary" property contains a summary of the message for display
+in a message index.
+
+Files
+""""""""""""
+
+Submitted files are represented by hyperdatabase
+nodes of class "file".  Like e-mail messages, the file content
+is stored in files outside the database,
+named after the file node designator (e.g. "file17").
+The "file" class has the definition::
+
+    hyperdb.Class(db, "file", user=hyperdb.Link("user"),
+                              name=hyperdb.String(),
+                              type=hyperdb.String())
+
+The "user" property indicates the user who submitted the
+file, the "name" property holds the original name of the file,
+and the "type" property holds the MIME type of the file as received.
+
+Issue Classes
+~~~~~~~~~~~~~
+
+All issues have the following standard properties:
+
+=========== ==========================
+Property    Definition
+=========== ==========================
+title       hyperdb.String()
+messages    hyperdb.Multilink("msg")
+files       hyperdb.Multilink("file")
+nosy        hyperdb.Multilink("user")
+superseder  hyperdb.Multilink("issue")
+=========== ==========================
+
+Also, two Date properties named "creation" and "activity" are
+fabricated by the Roundup database layer.  By "fabricated" we
+mean that no such properties are actually stored in the
+hyperdatabase, but when properties on issues are requested, the
+"creation" and "activity" properties are made available.
+The value of the "creation" property is the date when an issue was
+created, and the value of the "activity" property is the
+date when any property on the issue was last edited (equivalently,
+these are the dates on the first and last records in the issue's journal).
+
+Roundupdb Interface Specification
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The interface to a Roundup database delegates most method
+calls to the hyperdatabase, except for the following
+changes and additional methods::
+
+    class Database:
+        def getuid(self):
+            """Return the id of the "user" node associated with the user
+            that owns this connection to the hyperdatabase."""
+
+    class Class:
+        # Overridden methods:
+
+        def create(self, **propvalues):
+        def set(self, **propvalues):
+        def retire(self, nodeid):
+            """These operations trigger detectors and can be vetoed.  Attempts
+            to modify the "creation" or "activity" properties cause a KeyError.
+            """
+
+        # New methods:
+
+        def audit(self, event, detector):
+        def react(self, event, detector):
+            """Register a detector (see below for more details)."""
+
+    class IssueClass(Class):
+        # Overridden methods:
+
+        def __init__(self, db, classname, **properties):
+            """The newly-created class automatically includes the "messages",
+            "files", "nosy", and "superseder" properties.  If the 'properties'
+            dictionary attempts to specify any of these properties or a
+            "creation" or "activity" property, a ValueError is raised."""
+
+        def get(self, nodeid, propname):
+        def getprops(self):
+            """In addition to the actual properties on the node, these
+            methods provide the "creation" and "activity" properties."""
+
+        # New methods:
+
+        def addmessage(self, nodeid, summary, text):
+            """Add a message to an issue's mail spool.
+
+            A new "msg" node is constructed using the current date, the
+            user that owns the database connection as the author, and
+            the specified summary text.  The "files" and "recipients"
+            fields are left empty.  The given text is saved as the body
+            of the message and the node is appended to the "messages"
+            field of the specified issue.
+            """
+
+        def sendmessage(self, nodeid, msgid):
+            """Send a message to the members of an issue's nosy list.
+
+            The message is sent only to users on the nosy list who are not
+            already on the "recipients" list for the message.  These users
+            are then added to the message's "recipients" list.
+            """
+
+
+Default Schema
+~~~~~~~~~~~~~~
+
+The default schema included with Roundup turns it into a
+typical software bug tracker.  The database is set up like this::
+
+    pri = Class(db, "priority", name=hyperdb.String(), order=hyperdb.String())
+    pri.setkey("name")
+    pri.create(name="critical", order="1")
+    pri.create(name="urgent", order="2")
+    pri.create(name="bug", order="3")
+    pri.create(name="feature", order="4")
+    pri.create(name="wish", order="5")
+
+    stat = Class(db, "status", name=hyperdb.String(), order=hyperdb.String())
+    stat.setkey("name")
+    stat.create(name="unread", order="1")
+    stat.create(name="deferred", order="2")
+    stat.create(name="chatting", order="3")
+    stat.create(name="need-eg", order="4")
+    stat.create(name="in-progress", order="5")
+    stat.create(name="testing", order="6")
+    stat.create(name="done-cbb", order="7")
+    stat.create(name="resolved", order="8")
+
+    Class(db, "keyword", name=hyperdb.String())
+
+    Class(db, "issue", fixer=hyperdb.Multilink("user"),
+                       topic=hyperdb.Multilink("keyword"),
+                       priority=hyperdb.Link("priority"),
+                       status=hyperdb.Link("status"))
+
+
+(The "order" property hasn't been explained yet.  It
+gets used by the Web user interface for sorting.)
+
+The above isn't as pretty-looking as the schema specification
+in the first-stage submission, but it could be made just as easy
+with the addition of a convenience function like Choice
+for setting up the "priority" and "status" classes::
+
+    def Choice(name, *options):
+        cl = Class(db, name, name=hyperdb.String(), order=hyperdb.String())
+        for i in range(len(options)):
+            cl.create(name=option[i], order=i)
+        return hyperdb.Link(name)
+
+
+Detector Interface
+------------------
+
+Detectors are Python functions that are triggered on certain
+kinds of events.  The definitions of the
+functions live in Python modules placed in a directory set aside
+for this purpose.  Importing the Roundup database module also
+imports all the modules in this directory, and the ``init()``
+function of each module is called when a database is opened to
+provide it a chance to register its detectors.
+
+There are two kinds of detectors:
+
+1. an auditor is triggered just before modifying an node
+2. a reactor is triggered just after an node has been modified
+
+When the Roundup database is about to perform a
+``create()``, ``set()``, or ``retire()``
+operation, it first calls any *auditors* that
+have been registered for that operation on that class.
+Any auditor may raise a *Reject* exception
+to abort the operation.
+
+If none of the auditors raises an exception, the database
+proceeds to carry out the operation.  After it's done, it
+then calls all of the *reactors* that have been registered
+for the operation.
+
+Detector Interface Specification
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``audit()`` and ``react()`` methods
+register detectors on a given class of nodes::
+
+    class Class:
+        def audit(self, event, detector):
+            """Register an auditor on this class.
+
+            'event' should be one of "create", "set", or "retire".
+            'detector' should be a function accepting four arguments.
+            """
+
+        def react(self, event, detector):
+            """Register a reactor on this class.
+
+            'event' should be one of "create", "set", or "retire".
+            'detector' should be a function accepting four arguments.
+            """
+
+Auditors are called with the arguments::
+
+    audit(db, cl, nodeid, newdata)
+
+where ``db`` is the database, ``cl`` is an
+instance of Class or IssueClass within the database, and ``newdata``
+is a dictionary mapping property names to values.
+
+For a ``create()``
+operation, the ``nodeid`` argument is None and newdata
+contains all of the initial property values with which the node
+is about to be created.
+
+For a ``set()`` operation, newdata
+contains only the names and values of properties that are about
+to be changed.
+
+For a ``retire()`` operation, newdata is None.
+
+Reactors are called with the arguments::
+
+    react(db, cl, nodeid, olddata)
+
+where ``db`` is the database, ``cl`` is an
+instance of Class or IssueClass within the database, and ``olddata``
+is a dictionary mapping property names to values.
+
+For a ``create()``
+operation, the ``nodeid`` argument is the id of the
+newly-created node and ``olddata`` is None.
+
+For a ``set()`` operation, ``olddata``
+contains the names and previous values of properties that were changed.
+
+For a ``retire()`` operation, ``nodeid`` is the
+id of the retired node and ``olddata`` is None.
+
+Detector Example
+~~~~~~~~~~~~~~~~
+
+Here is an example of detectors written for a hypothetical
+project-management application, where users can signal approval
+of a project by adding themselves to an "approvals" list, and
+a project proceeds when it has three approvals::
+
+    # Permit users only to add themselves to the "approvals" list.
+
+    def check_approvals(db, cl, id, newdata):
+        if newdata.has_key("approvals"):
+            if cl.get(id, "status") == db.status.lookup("approved"):
+                raise Reject, "You can't modify the approvals list " \
+                    "for a project that has already been approved."
+            old = cl.get(id, "approvals")
+            new = newdata["approvals"]
+            for uid in old:
+                if uid not in new and uid != db.getuid():
+                    raise Reject, "You can't remove other users from the "
+                        "approvals list; you can only remove yourself."
+            for uid in new:
+                if uid not in old and uid != db.getuid():
+                    raise Reject, "You can't add other users to the approvals "
+                        "list; you can only add yourself."
+
+    # When three people have approved a project, change its
+    # status from "pending" to "approved".
+
+    def approve_project(db, cl, id, olddata):
+        if olddata.has_key("approvals") and len(cl.get(id, "approvals")) == 3:
+            if cl.get(id, "status") == db.status.lookup("pending"):
+                cl.set(id, status=db.status.lookup("approved"))
+
+    def init(db):
+        db.project.audit("set", check_approval)
+        db.project.react("set", approve_project)
+
+Here is another example of a detector that can allow or prevent
+the creation of new nodes.  In this scenario, patches for a software
+project are submitted by sending in e-mail with an attached file,
+and we want to ensure that there are text/plain attachments on
+the message.  The maintainer of the package can then apply the
+patch by setting its status to "applied"::
+
+    # Only accept attempts to create new patches that come with patch files.
+
+    def check_new_patch(db, cl, id, newdata):
+        if not newdata["files"]:
+            raise Reject, "You can't submit a new patch without " \
+                          "attaching a patch file."
+        for fileid in newdata["files"]:
+            if db.file.get(fileid, "type") != "text/plain":
+                raise Reject, "Submitted patch files must be text/plain."
+
+    # When the status is changed from "approved" to "applied", apply the patch.
+
+    def apply_patch(db, cl, id, olddata):
+        if cl.get(id, "status") == db.status.lookup("applied") and \
+            olddata["status"] == db.status.lookup("approved"):
+            # ...apply the patch...
+
+    def init(db):
+        db.patch.audit("create", check_new_patch)
+        db.patch.react("set", apply_patch)
+
+
+Command Interface
+-----------------
+
+The command interface is a very simple and minimal interface,
+intended only for quick searches and checks from the shell prompt.
+(Anything more interesting can simply be written in Python using
+the Roundup database module.)
+
+Command Interface Specification
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+A single command, roundup, provides basic access to
+the hyperdatabase from the command line::
+
+    roundup get [-list] designator[, designator,...] propname
+    roundup set designator[, designator,...] propname=value ...
+    roundup find [-list] classname propname=value ...
+
+TODO: more stuff here
+
+Property values are represented as strings in command arguments
+and in the printed results:
+
+- Strings are, well, strings.
+
+- Numbers are displayed the same as strings.
+
+- Booleans are displayed as 'Yes' or 'No'.
+
+- Date values are printed in the full date format in the local
+  time zone, and accepted in the full format or any of the partial
+  formats explained above.
+
+- Link values are printed as node designators.  When given as
+  an argument, node designators and key strings are both accepted.
+
+- Multilink values are printed as lists of node designators
+  joined by commas.  When given as an argument, node designators
+  and key strings are both accepted; an empty string, a single node,
+  or a list of nodes joined by commas is accepted.
+
+When multiple nodes are specified to the
+roundup get or roundup set
+commands, the specified properties are retrieved or set
+on all the listed nodes.
+
+When multiple results are returned by the roundup get
+or roundup find commands, they are printed one per
+line (default) or joined by commas (with the -list) option.
+
+Usage Example
+~~~~~~~~~~~~~
+
+To find all messages regarding in-progress issues that
+contain the word "spam", for example, you could execute the
+following command from the directory where the database
+dumps its files::
+
+    shell% for issue in `roundup find issue status=in-progress`; do
+    > grep -l spam `roundup get $issue messages`
+    > done
+    msg23
+    msg49
+    msg50
+    msg61
+    shell%
+
+Or, using the -list option, this can be written as a single command::
+
+    shell% grep -l spam `roundup get \
+        \`roundup find -list issue status=in-progress\` messages`
+    msg23
+    msg49
+    msg50
+    msg61
+    shell%
+    
+
+E-mail User Interface
+---------------------
+
+The Roundup system must be assigned an e-mail address
+at which to receive mail.  Messages should be piped to
+the Roundup mail-handling script by the mail delivery
+system (e.g. using an alias beginning with "|" for sendmail).
+
+Message Processing
+~~~~~~~~~~~~~~~~~~
+
+Incoming messages are examined for multiple parts.
+In a multipart/mixed message or part, each subpart is
+extracted and examined.  In a multipart/alternative
+message or part, we look for a text/plain subpart and
+ignore the other parts.  The text/plain subparts are
+assembled to form the textual body of the message, to
+be stored in the file associated with a "msg" class node.
+Any parts of other types are each stored in separate
+files and given "file" class nodes that are linked to
+the "msg" node.
+
+The "summary" property on message nodes is taken from
+the first non-quoting section in the message body.
+The message body is divided into sections by blank lines.
+Sections where the second and all subsequent lines begin
+with a ">" or "|" character are considered "quoting
+sections".  The first line of the first non-quoting 
+section becomes the summary of the message.
+
+All of the addresses in the To: and Cc: headers of the
+incoming message are looked up among the user nodes, and
+the corresponding users are placed in the "recipients"
+property on the new "msg" node.  The address in the From:
+header similarly determines the "author" property of the
+new "msg" node.
+The default handling for
+addresses that don't have corresponding users is to create
+new users with no passwords and a username equal to the
+address.  (The web interface does not permit logins for
+users with no passwords.)  If we prefer to reject mail from
+outside sources, we can simply register an auditor on the
+"user" class that prevents the creation of user nodes with
+no passwords.
+
+The subject line of the incoming message is examined to
+determine whether the message is an attempt to create a new
+issue or to discuss an existing issue.  A designator enclosed
+in square brackets is sought as the first thing on the
+subject line (after skipping any "Fwd:" or "Re:" prefixes).
+
+If an issue designator (class name and id number) is found
+there, the newly created "msg" node is added to the "messages"
+property for that issue, and any new "file" nodes are added to
+the "files" property for the issue.
+
+If just an issue class name is found there, we attempt to
+create a new issue of that class with its "messages" property
+initialized to contain the new "msg" node and its "files"
+property initialized to contain any new "file" nodes.
+
+Both cases may trigger detectors (in the first case we
+are calling the set() method to add the message to the
+issue's spool; in the second case we are calling the
+create() method to create a new node).  If an auditor
+raises an exception, the original message is bounced back to
+the sender with the explanatory message given in the exception.
+
+Nosy Lists
+~~~~~~~~~~
+
+A standard detector is provided that watches for additions
+to the "messages" property.  When a new message is added, the
+detector sends it to all the users on the "nosy" list for the
+issue that are not already on the "recipients" list of the
+message.  Those users are then appended to the "recipients"
+property on the message, so multiple copies of a message
+are never sent to the same user.  The journal recorded by
+the hyperdatabase on the "recipients" property then provides
+a log of when the message was sent to whom.
+
+Setting Properties
+~~~~~~~~~~~~~~~~~~
+
+The e-mail interface also provides a simple way to set
+properties on issues.  At the end of the subject line,
+``propname=value`` pairs can be
+specified in square brackets, using the same conventions
+as for the roundup ``set`` shell command.
+
+
+Web User Interface
+------------------
+
+The web interface is provided by a CGI script that can be
+run under any web server.  A simple web server can easily be
+built on the standard CGIHTTPServer module, and
+should also be included in the distribution for quick
+out-of-the-box deployment.
+
+The user interface is constructed from a number of template
+files containing mostly HTML.  Among the HTML tags in templates
+are interspersed some nonstandard tags, which we use as
+placeholders to be replaced by properties and their values.
+
+Views and View Specifiers
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+There are two main kinds of views: *index* views and *issue* views.
+An index view displays a list of issues of a particular class,
+optionally sorted and filtered as requested.  An issue view
+presents the properties of a particular issue for editing
+and displays the message spool for the issue.
+
+A view specifier is a string that specifies
+all the options needed to construct a particular view.
+It goes after the URL to the Roundup CGI script or the
+web server to form the complete URL to a view.  When the
+result of selecting a link or submitting a form takes
+the user to a new view, the Web browser should be redirected
+to a canonical location containing a complete view specifier
+so that the view can be bookmarked.
+
+Displaying Properties
+~~~~~~~~~~~~~~~~~~~~~
+
+Properties appear in the user interface in three contexts:
+in indices, in editors, and as filters.  For each type of
+property, there are several display possibilities.  For example,
+in an index view, a string property may just be printed as
+a plain string, but in an editor view, that property should
+be displayed in an editable field.
+
+The display of a property is handled by functions in
+a displayers module.  Each function accepts at
+least three standard arguments -- the database, class name,
+and node id -- and returns a chunk of HTML.
+
+Displayer functions are triggered by <display>
+tags in templates.  The call attribute of the tag
+provides a Python expression for calling the displayer
+function.  The three standard arguments are inserted in
+front of the arguments given.  For example, the occurrence of::
+
+    <display call="plain('status', max=30)">
+
+in a template triggers a call to::
+    
+    plain(db, "issue", 13, "status", max=30)
+
+
+when displaying issue 13 in the "issue" class.  The displayer
+functions can accept extra arguments to further specify
+details about the widgets that should be generated.  By defining new
+displayer functions, the user interface can be highly customized.
+
+Some of the standard displayer functions include:
+
+========= ====================================================================
+Function  Description
+========= ====================================================================
+plain     display a String property directly;
+          display a Date property in a specified time zone with an option
+          to omit the time from the date stamp; for a Link or Multilink
+          property, display the key strings of the linked nodes (or the
+          ids if the linked class has no key property)
+field     display a property like the
+          plain displayer above, but in a text field
+          to be edited
+menu      for a Link property, display
+          a menu of the available choices
+link      for a Link or Multilink property,
+          display the names of the linked nodes, hyperlinked to the
+          issue views on those nodes
+count     for a Multilink property, display
+          a count of the number of links in the list
+reldate   display a Date property in terms
+          of an interval relative to the current date (e.g. "+ 3w", "- 2d").
+download  show a Link("file") or Multilink("file")
+          property using links that allow you to download files
+checklist for a Link or Multilink property,
+          display checkboxes for the available choices to permit filtering
+========= ====================================================================
+
+
+Index Views
+~~~~~~~~~~~
+
+An index view contains two sections: a filter section
+and an index section.
+The filter section provides some widgets for selecting
+which issues appear in the index.  The index section is
+a table of issues.
+
+Index View Specifiers
+"""""""""""""""""""""
+
+An index view specifier looks like this (whitespace
+has been added for clarity)::
+
+    /issue?status=unread,in-progress,resolved&amp;
+        topic=security,ui&amp;
+        :group=+priority&amp;
+        :sort=-activity&amp;
+        :filters=status,topic&amp;
+        :columns=title,status,fixer
+
+
+The index view is determined by two parts of the
+specifier: the layout part and the filter part.
+The layout part consists of the query parameters that
+begin with colons, and it determines the way that the
+properties of selected nodes are displayed.
+The filter part consists of all the other query parameters,
+and it determines the criteria by which nodes 
+are selected for display.
+
+The filter part is interactively manipulated with
+the form widgets displayed in the filter section.  The
+layout part is interactively manipulated by clicking
+on the column headings in the table.
+
+The filter part selects the union of the
+sets of issues with values matching any specified Link
+properties and the intersection of the sets
+of issues with values matching any specified Multilink
+properties.
+
+The example specifies an index of "issue" nodes.
+Only issues with a "status" of either
+"unread" or "in-progres" or "resolved" are displayed,
+and only issues with "topic" values including both
+"security" and "ui" are displayed.  The issues
+are grouped by priority, arranged in ascending order;
+and within groups, sorted by activity, arranged in
+descending order.  The filter section shows filters
+for the "status" and "topic" properties, and the
+table includes columns for the "title", "status", and
+"fixer" properties.
+
+Associated with each issue class is a default
+layout specifier.  The layout specifier in the above
+example is the default layout to be provided with
+the default bug-tracker schema described above in
+section 4.4.
+
+Filter Section
+""""""""""""""
+
+The template for a filter section provides the
+filtering widgets at the top of the index view.
+Fragments enclosed in ``<property>...</property>``
+tags are included or omitted depending on whether the
+view specifier requests a filter for a particular property.
+
+Here's a simple example of a filter template::
+
+    <property name=status>
+        <display call="checklist('status')">
+    </property>
+    <br>
+    <property name=priority>
+        <display call="checklist('priority')">
+    </property>
+    <br>
+    <property name=fixer>
+        <display call="menu('fixer')">
+    </property>
+
+Index Section
+"""""""""""""
+
+The template for an index section describes one row of
+the index table.
+Fragments enclosed in ``<property>...</property>``
+tags are included or omitted depending on whether the
+view specifier requests a column for a particular property.
+The table cells should contain <display> tags
+to display the values of the issue's properties.
+
+Here's a simple example of an index template::
+
+    <tr>
+        <property name=title>
+            <td><display call="plain('title', max=50)"></td>
+        </property>
+        <property name=status>
+            <td><display call="plain('status')"></td>
+        </property>
+        <property name=fixer>
+            <td><display call="plain('fixer')"></td>
+        </property>
+    </tr>
+
+Sorting
+""""""""""""""
+
+String and Date values are sorted in the natural way.
+Link properties are sorted according to the value of the
+"order" property on the linked nodes if it is present; or
+otherwise on the key string of the linked nodes; or
+finally on the node ids.  Multilink properties are
+sorted according to how many links are present.
+
+Issue Views
+~~~~~~~~~~~
+
+An issue view contains an editor section and a spool section.
+At the top of an issue view, links to superseding and superseded
+issues are always displayed.
+
+Issue View Specifiers
+"""""""""""""""""""""
+
+An issue view specifier is simply the issue's designator::
+
+    /patch23
+
+
+Editor Section
+""""""""""""""
+
+The editor section is generated from a template
+containing <display> tags to insert
+the appropriate widgets for editing properties.
+
+Here's an example of a basic editor template::
+
+    <table>
+    <tr>
+        <td colspan=2>
+            <display call="field('title', size=60)">
+        </td>
+    </tr>
+    <tr>
+        <td>
+            <display call="field('fixer', size=30)">
+        </td>
+        <td>
+            <display call="menu('status')>
+        </td>
+    </tr>
+    <tr>
+        <td>
+            <display call="field('nosy', size=30)">
+        </td>
+        <td>
+            <display call="menu('priority')>
+        </td>
+    </tr>
+    <tr>
+        <td colspan=2>
+            <display call="note()">
+        </td>
+    </tr>
+    </table>
+
+As shown in the example, the editor template can also
+request the display of a "note" field, which is a
+text area for entering a note to go along with a change.
+
+When a change is submitted, the system automatically
+generates a message describing the changed properties.
+The message displays all of the property values on the
+issue and indicates which ones have changed.
+An example of such a message might be this::
+
+    title: Polly Parrot is dead
+    priority: critical
+    status: unread -> in-progress
+    fixer: (none)
+    keywords: parrot,plumage,perch,nailed,dead
+
+If a note is given in the "note" field, the note is
+appended to the description.  The message is then added
+to the issue's message spool (thus triggering the standard
+detector to react by sending out this message to the nosy list).
+
+Spool Section
+"""""""""""""
+
+The spool section lists messages in the issue's "messages"
+property.  The index of messages displays the "date", "author",
+and "summary" properties on the message nodes, and selecting a
+message takes you to its content.
+
+
+Deployment Scenarios
+--------------------
+
+The design described above should be general enough
+to permit the use of Roundup for bug tracking, managing
+projects, managing patches, or holding discussions.  By
+using nodes of multiple types, one could deploy a system
+that maintains requirement specifications, catalogs bugs,
+and manages submitted patches, where patches could be
+linked to the bugs and requirements they address.
+
+
+Acknowledgements
+----------------
+
+My thanks are due to Christy Heyl for 
+reviewing and contributing suggestions to this paper
+and motivating me to get it done, and to
+Jesse Vincent, Mark Miller, Christopher Simons,
+Jeff Dunmall, Wayne Gramlich, and Dean Tribble for
+their assistance with the first-round submission.
+
+Changes to this document
+------------------------
+
+- Added Boolean and Number types
+- Added section Hyperdatabase Implementations
+- "Item" has been renamed to "Issue" to account for the more specific nature
+  of the Class.
+
+