Mercurial > p > roundup > code
changeset 1242:3d0158c8c32b maint-0.5
subsumed by other doc
| author | Richard Jones <richard@users.sourceforge.net> |
|---|---|
| date | Mon, 30 Sep 2002 01:17:10 +0000 |
| parents | bc0ffa0cd7ac |
| children | 83f33642d220 |
| files | BUILD.txt CHANGES.txt MANIFEST.in TODO.txt cgi-bin/roundup.cgi doc/.cvsignore doc/FAQ.txt doc/Makefile doc/announcement.txt doc/customizing.txt doc/design.txt doc/developers.txt doc/getting_started.txt doc/index.txt doc/installation.txt doc/upgrading.txt doc/user_guide.txt frontends/ZRoundup/ZRoundup.py frontends/ZRoundup/__init__.py roundup/__init__.py roundup/admin.py roundup/backends/__init__.py roundup/backends/back_anydbm.py roundup/backends/back_bsddb3.py roundup/backends/back_gadfly.py roundup/backends/back_metakit.py roundup/backends/back_sqlite.py roundup/backends/portalocker.py roundup/backends/rdbms_common.py roundup/cgi/PageTemplates/PythonExpr.py roundup/cgi/PageTemplates/TALES.py roundup/cgi/TAL/TALGenerator.py roundup/cgi/TAL/TALInterpreter.py roundup/cgi/__init__.py roundup/cgi/client.py roundup/cgi/templating.py roundup/date.py roundup/hyperdb.py roundup/mailgw.py roundup/password.py roundup/roundupdb.py roundup/scripts/roundup_server.py roundup/templates/classic/config.py roundup/templates/classic/dbinit.py roundup/templates/classic/detectors/statusauditor.py roundup/templates/classic/html/issue.index roundup/templates/classic/html/issue.item roundup/templates/classic/html/issue.search roundup/templates/classic/html/query.item roundup/templates/classic/html/style.css roundup/templates/classic/html/user.index roundup/templates/classic/html/user.item roundup/templates/classic/interfaces.py roundup/templates/minimal/config.py roundup/templates/minimal/dbinit.py roundup/templates/minimal/html/style.css roundup/templates/minimal/html/user.index roundup/templates/minimal/interfaces.py setup.py test/test_dates.py test/test_db.py test/test_locking.py test/test_mailgw.py test/test_mailsplit.py |
| diffstat | 63 files changed, 0 insertions(+), 25769 deletions(-) [+] |
line wrap: on
line diff
--- a/BUILD.txt Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,55 +0,0 @@ - Roundup - ======= - -1. Building Releases -==================== - -Roundup is currently a source-only release - it has no binary components. I -want it to stay that way, too. - -This means that we only need to ever build source releases. This is done by -running: - - 0. Edit setup.py and doc/announcement.txt to reflect the new version and - appropriate announcments. - 1. python setup.py clean --all - 2. Edit setup.py to ensure that all information therein (version, contact - information etc) is correct. - 3. python setup.py sdist --manifest-only - 4. Check the MANIFEST to make sure that any new files are included. If - they are not, edit MANIFEST.in to include them. "Documentation" for - MANIFEST.in may be found in disutils.filelist._parse_template_line. - 5. python setup.py sdist - (if you find sdist a little verbose, add "--quiet" to the end of the - command) - -So, those commands in a nice, cut'n'pasteable form :) -python setup.py clean --all -python setup.py sdist --manifest-only -python setup.py sdist --quiet - -or, for the sad RedHat users: -python2 setup.py clean --all -python2 setup.py sdist --manifest-only -python2 setup.py sdist --quiet - - -2. Distributing Releases -======================== - -Once a release is built, follow these steps: - 1. FTP the tar.gz from the dist directory to to the "incoming" directory on - "upload.sourceforge.net". - 2. Make a quick release at: - http://sourceforge.net/project/admin/qrs.php?package_id=&group_id=31577 - 3. Add a news item at: - https://sourceforge.net/news/submit.php?group_id=31577 - using the top of doc/announcement.txt - 4. Send doc/announcement.txt to python-announce@python.org - 5. Notify any other news services as appropriate... - - -3. Author -========= -richard@users.sourceforge.net -
--- a/CHANGES.txt Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,767 +0,0 @@ -This file contains the changes to the Roundup system over time. The entries -are given with the most recent entry first. - -2002-09-?? 0.5.0 ???? -- handling of None for Date/Interval/Password values in export/import -- handling of journal values in export/import -- password edit now has a confirmation field -- registration error punts back to register page -- gadfly backend now handles changes to the schema - but only one property - at a time -- cgi.client base URL is now obtained from the config TRACKER_WEB -- request.url has gone away - there's too much magic in trying to figure - what it should be -- cgi-bin script redirects to https now if the request was https -- FileClass "content" property wasn't being returned by getprops() in most - backends -- we now verify instance attributes on instance open and throw a useful error - if they're not all there -- sf 611217 ] menu() has problems when labelprop==None -- verify contents of tracker module when the tracker is opened -- many performance improvements in *dbm and sql backends -- mailgw was missing an "import sys" -- setup now installs scripts with python -O flag, doubling performance in some - cases (there's a lot of __debug__ use) -- fix :required for Link menus -- import wasn't setting the ID to maxid+1 -- added getItem to HTMLClass so you can access arbitrary items in templates -- index filtering form values may now be key values too -- replaced the content() callback ickiness with Page Template macro usage -- changed the default CSS style to be less offensive to some ;) -- better handling of Page Template compilation errors -- handle multiple unrelated indexed classes -- #614188 ] Exception in mailgw.py -- #613310 ] traceback on onexistant items -- #613291 ] typos in nosy list -- handle stupid mailers that QUOTE their Re; 'Re: "[issue1] bla blah"' -- giving a user a Role that doesn't exist doesn't break stuff any more -- revamped user guide, customisation guide, added maintenance guide -- merge Zope Collector #538 fix from ZPT CVS trunk (path expressions with a - non-path final alternate no longer try to call a value returned by that - alternate) -- merge Zope Collector #573 fix from ZPT CVS trunk -- merge Zope Collector #580 fix from ZPT CVS trunk -- added "crypt" password encoding and ability to set password with - already encrypted password through roundup-admin - - -2002-09-13 0.5.0 beta2 -- all backends now have a .close() method, and it's used everywhere -- fixed bug in detectors __init__ -- switched the default issue item display to only show issue summary - (added instructions to doc to make it display entire content) -- MANIFEST.in was missing a lot of template files -- added generic item editing -- much nicer layout of template rendering errors -- added context/is_edit_ok and context/is_view_ok convenience methods and - implemented use of them in the classic template - - -2002-09-11 0.5.0 beta1 -Fixed: -- #576086 ] dumb copying mistake (frontends/ZRoundup.py) -- installation instructions now mention "python2" in "testing your python". -- made the unit tests run again - they were quite b0rken -- #571170 ] gdbm deadlock -- #576241 ] MultiLink problems in parsePropsFromForm -- fixed the date module so that Date(". - 2d") works -- web forms may now unset Link values (like assignedto) -- cleanup: moved roundup.templatebuilder to roundup.templates.builder -- instance __init__ no longer silently traps dbinit import errors - -Feature: -- new backend for metakit (thanks Gordon McMillan) -- new backend for gadfly (it's as done as it's going to get) -- further split the dbm backends from the core code, allowing easier - non-dict-like backends (eg metakit, RDB) -- implemented and used the new access control mechanisms (Permissions, Roles) - (see doc/security.txt) -- switched templating to use Zope's PageTemplates (yay!) -- switched to sessions for web authentication -- added Boolean and Number types -- fixed the journal bloat -- updated design document for new access controls -- updated customisation document, including more examples -- entire database export and import (incl files) -- better mailgw help message (feature request #558562) -- re-enabled link backrefs from messages (feature request #568714) -- the page layout is now templatable -- re-worked cgi interface to abstract out the explicit "issue" interface -- have index page handle mid-page errors better so header and footer are - still visible -- we handle "not found", access and item page render errors better -- fixed double-submit by having new-item-submit redirect at end -- daemonify roundup-server (fork, logfile, pidfile) -- modify cgitb to display PageTemplate errors better -- rename to "instance" to "tracker" -- have roundup.cgi pick up tracker config from the environment -- revamped look and feel in web interface -- cleaned up stylesheet usage -- several bug fixes and documentation fixes -- added is_retired test to hyperdb.Class -- added capability to save queries: - - a query Class with name, klass (to search) and url (query string) - properties - - a Multilink to query on user called queries - - html templates for query, and a list of queries in user.item - - search form has Save button & name input - - saved queries put in menu in pagehead - - for migration, none of the above is required and old behavior preserved. - - showquery translates search form <-> query string -- cleaned up the indexer code: - - it splits more words out - - removed code we'll never use (roundup.roundup_indexer has the full - implementation, and replaces roundup.indexer) - - only index text/plain and rfc822/message (ideas for other text formats to - index are welcome) - - added simple unit test for indexer. Needs more tests for regression. - - all String properties may now be indexed too. Currently there's a bit of - "issue" specific code in the actual searching which needs to be - addressed. In a nutshell: - + pass 'indexme="yes"' as a String() property initialisation arg, eg: - file = FileClass(db, "file", name=String(), type=String(), - comment=String(indexme="yes")) - + the comment will then be indexed and be searchable, with the results - related back to the issue that the file is linked to - - as a result of this work, the FileClass has a default MIME type that may - be overridden in a subclass, or by the use of a "type" property as is - done in the default templates. - - the regeneration of the indexes (if necessary) is done once the schema is - set up in the dbinit. - - new "reindex" command in roundup-admin used to force regeneration of the - index -- added email display function - mangles email addrs so they're not so easily - scraped from the web -- switched to using a session-based web login -- made mailgw handle set and modify operations on multilinks (bug #579094) -- fixed the journal bloat from multilink changes - we just log the add or - remove operations, not the whole list - - -2002-06-24 0.4.2 -Fixed: -- Cleaned up the hyperdb unit tests. -- Applied patch from Andrew W. Nosenko to give nicer Unauthorised message - when anonymous user tries to edit. Should've been applied in 0.4.2pr1. Oops. -- Added more detailed note to MIGRATION regarding the detectors changes. - - -2002-06-19 0.4.2pr1 -Feature: -- added a "detectors" directory for people to put their useful auditors and - reactors in. Note - the roundupdb.IssueClass.sendmessage method has been - split and renamed "nosymessage" specifically for things like the nosy - reactor, and "send_message" which just sends the message. -- link() htmltemplate function now has a "showid" option for links and - multilinks. When true, it only displays the linked node id as the anchor - text. The link value is displayed as a tooltip using the title anchor - attribute. - To use in eg. the superseder field, have something like this: - <td> - <display call="field('superseder', showid=1)"> - <display call="classhelp('issue', 'id,title', label='list', width=500)"> - <property name="superseder"> - <br>View: <display call="link('superseder', showid=1)"> - </property> - </td> -- stripping of the email message body can now be controlled through the - config variables EMAIL_KEEP_QUOTED_TEXT and EMAIL_LEAVE_BODY_UNCHANGED. -- all database files created are now group readable and writable. -- added option to automatically add the authors and recipients of messages - to the nosy lists with the options ADD_AUTHOR_TO_NOSY (default 'new') and - ADD_RECIPIENTS_TO_NOSY (default 'new'). These settings emulate the current - behaviour. Setting them to 'yes' will add the author/recipients to the nosy - on messages that create issues and followup messages. -- reverting to dates for intervals > 2 months sucks -- changed the default message list in issues to display the message body -- applied patch #558876 ] cgi client customization -- split instance initialisation into two steps, allowing config changes - before the database is initialised. -- don't create an empty message on email issue creation if the email is empty -- may now display additional fields in Multilink form menus -- #541941 ] changing multilink properties by mail -- #526730 ] search for messages capability -- #505180 ] split MailGW.handle_Message - - also changed cgi client since it was duplicating the functionality - -Fixed: -- stop sending blank (whitespace-only) notes -- cleanup of serialisation for database storage -- node ids are now generated from a lockable store - no more race conditions -- sorting was applied to all nodes of the MultiLink class instead of - to the nodes that are actually linked to in the "field" template - function. This adds about 20+ seconds in the display of an issue if - your database has a 1000 or more issues in it. -- added missing documentation for a few of the config option values -- file upload broke if you didn't supply a change note -- fixed SCRIPT_NAME in ZRoundup for instances not at top level of Zope - (thanks dman) -- fixed some sorting issues that were breaking some unit tests under py2.2 -- mailgw test output dir was confusing the init test (but only on 2.2 *shrug*) -- node caching now works, and gives a small boost in performance -- #449374 ] re-enable bsddb3 backend - bsddb3 backend now works, reinstating -- #551483 ] assignedto in Client.make_index_link -- made backends.__init__ be more specific about which ImportErrors it really - wants to ignore -- fixed the example addresses in the templates to use correct example domains -- cleaned out the template stylesheets, removing a bunch of junk that really - wasn't necessary (font specs, styles never used) and added a style for - message content -- build htmlbase if tests are run using CVS checkout -- #565979 ] code error in hyperdb.Class.find -- #565996 ] The "Attach a File to this Issue" fails -- #564271 ] find() and new properties -- #562130 ] cookie path generated from ZRoundup was wrong in some situations -- remove CR characters embedded in messages (ZRoundup) -- properly quote the email address and "real name" in all situations using the - 'email' module if it is available and 'rfc822' otherwise -- #565992 ] if ISSUE_TRACKER_WEB doesn't have the trailing '/', add it -- use the rfc822 module to ensure that every (oddball) email address and - real-name is properly quoted -- #558867 ] ZRoundup redirect /instance requests to /instance/ -- #569415 ] {version} -- #569178 ] type error - was fixed as part of the general cleanup of reactors - - -2002-03-25 - 0.4.1 -Feature: -- use blobfiles in back_anydbm which is used in back_bsddb. - change test_db as dirlist does not work for subdirectories. - ATTENTION: blobfiles now creates subdirectories for files. -- add module blobfiles in backends with file access functions. -- roundup db catch only IOError in getfile. -- roundup db catches retrieving not existing files. -- #503204 ] mailgw needs a default class - - partially done - the setting of additional properties can wait for a - better configuration system. -- Alternate email addresses are now available for users. See the MIGRATION - file for info on how to activate the feature. -- #511168 ] Web interface: Adding new products - Classes that don't provide template html get a default edit interface now: - - access using the admin "class list" interface - - limited to admin-only - - requires the csv module from object-craft (url given if it's missing) -- Added popup help for classes using the classhelp html template function. - - add <display call="classhelp('priority', 'id,name,description')"> - to an item page, and it generates a link to a popup window which displays - the id, name and description for the priority class. The description - field won't exist in most installations, but it will be added to the - default templates. -- #517734 ] web header customisation is obscure -- All messages sent to the nosy list are now encoded as - quoted-printable before they are sent. -- Fixed display of mutlilink properties when using the template - functions, menu and plain. - -Fixed: -- Clean up mail handling, multipart handling. -- respect encodings in non multipart messages. -- makeHtmlBase: re.sub under python 2.2 did not replace '.', string.replace - does it. -- preamble in tepmlateBuilder mentioned htmldata -- mailgw checks encoding on first part too. -- #511586 ] unittest FAIL: testReldate_date -- Added a uniquely Roundup header to email, "X-Roundup-Name" -- All forms now have "double-submit" protection when Javascript is enabled - on the client-side. -- #516883 ] mail interface + ANONYMOUS_REGISTER -- #516854 ] "My Issues" and redisplay -- #517906 ] Attribute order in "View customisation" -- #514854 ] History: "User" is always ticket creator -- wasn't handling cvs parser feeding correctly -- fixed some problems in date calculations (calendar.py doesn't handle over- - and under-flow). Also, hour/minute/second intervals may now be more than - 99 each. -- #527416 ] roundup-admin uses undefined value -- #527503 ] unfriendly init blowup when parent dir - (also handles UsageError correctly now in init) -- #524129 ] roundup-admin gets python path wrong - - -2002-01-24 - 0.4.0 -Feature: -- much nicer history display (actualy real handling of property types etc) -- journal entries for link and mutlilink properties can be switched on or - off -- properties in change note are now sorted -- you can now use the roundup-admin tool pack the database - -Fixed: -- the mail gateway now responds with an error message when invalid values - for arguments are specified for link or mutlilink properties -- modified unit test to check nosy and assignedto when specified as arguments -- handle attachments with no name (eg tnef) -- fixed setting nosy as argument in subject line -- fixed back_bsddb so it passed the journal tests -- fixed status changes in mail gateway (eg. unread -> chatting) -- we'll actually distribute the frontends directory now, as advertised... -- handle stripping of "AW:" from subject line -- htmltemplate list() wasn't sorting... -- unit tests for html templating (and re-enabled the listbox field for - multilinks) -- allow abbreviation of "help" in admin tool too. -- run_tests testReldate_date failed if LANG is 'german' -- mailgw failures (unexpected ones) are forwarded to the roundup admin - - -2002-01-16 - 0.4.0b2 -Fixed: -- #495392 ] empty nosy -patch -- #500574 ] messageid must have format <part1@part2> -- fixed some problems with web editing and change detection -- mail splitting wasn't detecting responses in the same "section" as quoted - text -- missed a "from i18n import _" in date.py -- #501690 ] MIGRATION.txt incomplete -- #502342 ] pipe interface -- #502437 ] rogue reactor and unittest -- re-enabled dumbdbm when using python >2.1.1 (ie 2.1.2, 2.2) -- changed all config accesses so they access either the instance or the - config attriubute on the db. This means that all config is obtained from - instance_config instead of the mish-mash of classes. This will make - switching to a ConfigParser setup easier too, I hope. -- #502951 ] adding new properties to old database -- #502953 ] nosy-like treatment of other multilinks -- #503164 ] create and passwords -- plain rendering of links in the htmltemplate now generate a hyperlink to - the linked node's page. -- #503330 ] ANONYMOUS_REGISTER now applies to mail -- #503353 ] setting properties in initial email -- #502956 ] filtering by multilink not supported -- #503340 ] creating issue with [asignedto=p.ohly] -- #502949 ] index view for non-issues and redisplay -- #503793 ] changing assignedto resets nosy list -- lots of date/interval related changes: - - more relaxed date format for input - - handle None for date/interval properties - - -2002-01-08 - 0.4.0b1 -Feature: -- Added INSTANCE_NAME to configuration - used in web and email to identify - the instance. -- Added EMAIL_SIGNATURE_POSITION to indicate where to place the roundup - signature info in e-mails. -- Some more flexibility in the mail gateway and more error handling. -- Login now takes you to the page you back to the were denied access to. -- Admin user now can has a user index link on their web interface. -- We now have basic transaction support. Information is only written to - the database when the commit() method is called. Only the anydbm and - bsddb3 backends are modified in this way - the bsddb3 backend needs a - lot more work anyway... - - the CGI and mailgw automatically commit() at the end of processing a - single transaction - - the admin tool requires an explicit "commit" - it will prompt at exit - if there are unsaved changes. A "rollback" removes all changes made - during the session (up to the last commit). -- Added the "display" command to the admin tool - displays a node's values -- Message author's name appears in From: instead of roundup instance name - (which still appears in the Reply-To:) -- Added a Zope frontend for roundup. -- Centralised the python version check code, bumped version to 2.1.1 (really - needs to be 2.1.2, but that isn't released yet :) -- much better attaching of erroneous messages in the mail gateway -- #496356 ] Use threading in messages - This adds the tracking of messages by message-id and allows threading - using in-reply-to. Most e-mail clients support threading using this - feature, and we hope to add support for it to the web gateway. - -Fixed: -- Lots of bugs, thanks Roché and others on the devel mailing list! -- login_action and newuser_action return values were being ignored -- Woohoo! Found that bloody re-login bug that was killing the mail - gateway. -- Fixed login/registration forwarding the user to the right page (or not, - on a failure) -- We now use weakrefs in the Classes to keep the database reference, so - the close() method on the database is no longer needed. -- #487480 ] roundup-server -- #487476 ] INSTALL.txt -- #489760 ] [issue] only subject -- fixed doc/index.html to include the quoting in the mail alias. -- fixed the backends __init__ so we can pydoc the backend modules -- web i/f reports "note added" if there are no changes but a note is entered -- we were assuming database files created by anydbm had the same name, but - this is not the case for dbm. We now perform a much better check _and_ - cope with the anydbm implementation module changing too! -- envelope-from is now set to the roundup-admin and not roundup itself so - delivery reports aren't sent to roundup (thanks Patrick Ohly) -- #495400 ] entering blanks - Values with spaces are now accepted in roundup-admin - check the long help - for details. -- #496360 ] table width does not work -- detectors were being registered multiple times -- added tests for mailgw - - -2001-11-23 - 0.3.0 -Feature: -- #467129 ] Lossage when username=e-mail-address -- #473123 ] Change message generation for author -- MailGW now moves 'resolved' to 'chatting' on receiving e-mail for an issue. -- Added Structured Text rendering to htmltemplate, thanks Brad Clements. -- Added CGI configuration via env vars (see roundup.cgi for details) -- "roundup.cgi" is now installed to "<python-prefix>/share/roundup/cgi-bin" -- roundup-admin now accepts abbreviated commands (eg. l = li = lis = list) -- roundup-mailgw now supports unix mailbox and POP as sources of mail. -- roundup-admin now handles all hyperdb exceptions -- users may attach files to issues (and support in ext) through the web now -- incorporated patch from Roch'e Compaan implementing attachments in nosy - e-mail -- added a target version field to the extended issue schema -- added dummy hooks for I18N and some preliminary (test) markup of - translatable messages - -Fixed: -- Fixed a bug in HTMLTemplate changes. -- 'unread' to 'chatting' automagic status change was b0rken. -- Anonymous user lockout wasn't working. -- roundup-server now works on Windows, thanks Juergen Hermann. -- Fixed install documentation, also thanks Juergen Hermann. -- Fixed some URL issues in roundup.cgi, again thanks Juergen Hermann. -- bug #475347 ] WindowsError still not caught (patch from Juergen Hermann) -- bug #474749 ] indentations lost -- bug #477104 ] HTML tag error in roundup-server -- bug #477107 ] HTTP header problem -- bug #477687 ] conforming html -- bug #474372 ] Netscape 4.77 do not render Support form -- bug #477685 ] base64.decodestring breaks -- bug #477837 ] lynx does not like the cookie -- bug #477892 ] Password edit doesn't fix login cookie -- newuser_action now presents error messages rather than tracebacks. -- bug #479511 ] mailgw to pop -- bug #479508 ] roundup-admin crash on wrong class -- bad error report in hyperdb -- roundup.mailgw now handles errors on the set() and create() at the end - of processing -- roundup.mailgw also handles messages that are passed to it that don't - contain a From: line - apparently some POP servers can do this. It punts - an error message to the roundup admin. -- fixed nosy reaction and author copy handling -- errors in nosy reaction will be propogated now (were effectively being - squashed) -- re-open the database as the author in mail handling -- missing "return" in filter_section (thanks Roch'e Compaan) - - -2001-10-23 - 0.3.0 pre 3 -Feature: -- MailGW now moves 'unread' to 'chatting' on receiving e-mail for an issue. -- feature #473127: Filenames. I modified the file.index and htmltemplate - source so that the filename is used in the link and the creation - information is displayed. - Admin Tool (roundup-admin): - - Interactive mode for running multiple (independant at present) commands. - - Tabular display of nodes. - - Import and export via colon-separated files. - -Changed: -- re-organised the html templating code. Fixed some bugs, probably - introduced some more. Hopefully not too many. - -Fixed: -- Stand-alone server now has a configurable setuid user. -- CGI interface wasn't handling checkboxes at all. -- Fixed quopri usage in mailgw from bug reports on mailing list. -- Remove the "freshen" command from the roundup-admin tool. -- Catch errors in login - no username or password supplied. -- Fixed editing of password (Password property type) thanks Roch'e Compaan. -- Fixed grouping of non-str properties thanks Roch'e Compaan. -- bug #473121: The customisation view and filters (CGI interface view - customisation section may now be hidden (patch from Roch'e Compaan.) -- bug #473122: Issue id sorting (hyperdb sorts strings-that-look-like-numbers - as numbers now. -- bug #473124: UI inconsistency with Link fields. - This also prompted me to fix a fairly long-standing usability issue - - that of being able to turn off certain filters. -- bug #473125: Paragraph in e-mails -- bug #473126: Sender unknown -- bug #473130: Nosy list not set correctly - - -2001-10-11 - 0.3.0 pre 2 -Fixed: -- Hyperdatabase was inserting empty strings instead of None for missing - property values. This broke a lot of things. - - -2001-10-10 - 0.3.0 pre 1 -Feature: -- roundup-admin create now prompts for property info if none is supplied - on the command-line. -- hyperdb Class getprops() method may now return only the mutable - properties. -- CGI interfaces now generate a top-level index of their known instances. - -Changed: -- Login now uses cookies, which makes it a whole lot more flexible. We can - now support anonymous user access (read-only, unless there's an - "anonymous" user, in which case write access is permitted). Login - handling has been moved into cgi_client.Client.main() -- The "extended" schema is now the default in roundup init. -- The schemas have had their page headings modified to cope with the new - login handling. Existing installations should copy the interfaces.py - file from the roundup lib directory to their instance home. -- Passwords are now encoded by default (except exising databases which - will only be encoded when the passwords are changed). The scheme used - at the moment is SHA - but the code is flexible enough to take any - number of encoding systems. -- The roundup-admin tool always operates as the "admin" user now. Database - protection should be achieved using file system protections (see the - documentation for details.) - -Fixed: -- Incorrectly had a Bizar Software copyright on the cgitb.py module from - Ping - has been removed. -- Pretty time interval wasn't handling > 1 month properly. -- Generation of links to Link/Multilink in indexes. (thanks Hubert Hoegl) -- AssignedTo wasn't in the "classic" schema's item page. -- Fixed a whole bunch of places in the CGI interface where we should have - been returning Not Found instead of throwing an exception. -- Fixed a deviation from the spec: trying to modify the 'id' property of - an item now throws an exception. -- The plain() template function now html-escapes the content. -- Change message was stuffing up for multilinks with no key property. - - - --------------- - -2001-08-30 - 0.2.8 -Fixed: -- Wasn't handling unguessable mime types for file uploads. -- Missing import in mailgw. - - -2001-08-29 - 0.2.7 -Feature: -- Text searches are now case insensitive. All forms of text search use - regular expressions now. - -Fixed: -- Had another 2.1-ism in the unit tests -- Made the mail parser a little more robust w.r.t missing Subject: - (both thanks Mikhail Sobolev) -- Missed some isFooType usages (thanks Mikhail Sobolev for spotting them) -- Reverted back to sending change messages to the web editor of a node so - that the change note message is actually genrated. -- CGI interface wasn't generating correct change messages. -- Notes entered during a change are saved to the messages list even if - there's no nosy list. No message is generated if there's no nosy list and - there's no change note (since it would just duplicates the journal). -- Completely removed the bsddb3 module from the tests - will be reinstated - when the http://bsddb.sourceforge.net/'s bugs #439959 and #456408 are - dealt with. One is fixed in CVS, the other pending. - - -2001-08-08 - 0.2.6 -Note: -- Roundup is now released under the same terms as the Python License. - -Feature: -- Added tests for instance initialisation. No more releasing the software - with bugs in roundup.init! -- Now bundling unittest with the package so that python 2.0 users can use - the tests. -- Much better error handling and messages generated by the mail gateway. - -Fixed: -- Implemented correct mail splitting. Added unit tests. Also snips - signatures now too. -- Bug #447671 - typo in roundup/init.py -- Changed date.Date to use regular string formatting instead of strftime - - win32 seems to have problems with %T and no hour... or something... -- Bug #448484 - now catching correct exception from makedirs. -- Instances are now opened by a special function that generates a unique - module name for the instances on import time. - - -2001-08-03 - 0.2.5 -Note: -- The bsddb3 module has a bug that renders it non-functional. Users should - select the anydbm or bsddb backend instead. - -Fixed: -- Python 2.0 does not contain the unittest module. The setup.py module now - checks for unittest before attempting to run the unit tests. - - -2001-08-03 - 0.2.4 -Features: -- Added ability for cgi newblah forms to indicate that the new node - should be linked somewhere. -- Added time logging and file uploading to the templates. -- Added "My Issues" and "My Support" to extended template. Changed "Your - Details" to "My Details". Changed the "New Foo" links to "Add Foo". - Added links for unassigned support and issues. Generally reorganised and - cleanup the header up. -- Changed the order of the information in the message generated by web edits. -- Extended the range of intervals that are pretty-printed before actual dates - are displayed. -- Added more BUILD instructions including the "clean" command to force - rebuild. -- Web edit messages aren't sent to the person who did the edit any more. No - message is generated if they are the only person on the nosy list. -- Roundupdb now appends "mailing list" information to its messages which - include the e-mail address and web interface address. Templates may - override this in their db classes to include specific information (support - instructions, etc). - -Fixed: -- Argument handling for the roundup-admin find command. -- Handling of summary when no note supplied for newblah. Again. -- Detection of no form in htmltemplate Field display. -- Checklist html template command was setting wrong name. -- 2.1-specific gmtime() (no arg) call in roundup.date. (thanks Paul Wright) -- mailgw was making naughty assumptions about the schema of the classes it - was creating nodes for. -- remove the $Foo$ from the HTML files stored in the htmlbase modules. -- Instance import now imports the instance using imp.load_module so that - we can have instance homes of "roundup" or other existing python package - names. - - -2001-07-30 - 0.2.3 -Big change: -- I've split off the support class from the issue class in "extended". - Anyone who has any support entries, sorry. It should be possible to - write a scipt that moves the entries over pretty easily. If this causes - you pain, I'll do so. You'll want to update your instance with the new - code in "extended" either way. - -Features: -- Added the unit tests to the start of setup.py so they're run whenever - we do anything distutils'y. -- Added nicer prompting to the roundup-admin "init" command. -- Actually, the roundup-admin code is totally revamped, and has command - help and better command-line arg handling. -- The cgi_client.Client base class now reflects the structure of "classic" - rather than "extended" since "classic" is more of a "base" template. -- Added more DB to test. Skips tests where imports fail. - -Fixed: -- One of the tests in test_date had the wrong expected result. -- Fixed IssueClass so that superseders links to its classname rather than - hard-coded to "issue". -- templatebuilder was catching IOError instead of OSError. -- The cgi_client newblah method wasn't detecting the __note form field - properly. -- The History command in htmltemplate didn't handle a new node (None - nodeid) properly. - - -2001-07-29 - 0.2.2 -Features: -- Added implementation.txt to the doc directory. Contains implementation - notes specific to this implementations of Roundup. -- Cleaned up mailgw some (subclass Message for getPart) and added some - tests for multipart splitting. -- Better checking for html dir in templatebuilder. -- Base hyperdb.Class now fakes the "id" property. -- Made the classic roundup look more like the original prototype. -- Made cgi_client and templating slightly more generic. -- Moved some code around in cgi_client allowing for subclassing to change - behaviour. -- Added the fabricated property "id" to all hyperdb classes. -- Cleanup of the link label generation (new method on hyperdb.Class to do - it). - -Fixed: -- Everything uses errno module now to check errno values. -- New issue form handles lack of note better now. -- HTML templating uses section-bar style for index group headers now. -- Fixed problem in link display when Link value is None. -- Form handling in cgi client wasn't propogating through the previous - query elements. -- Fixed sort arguments generated for column headings so sorting can be - changed now. - - -2001-07-28 - 0.2.1 -Features: -- Added docstring to roundup package so pydoc reports useful information. -- Added the roundup 1 software carpentry submission HTML to the doc - directory as "overview.html". - -Fixes: -- Fixed bug in init command - templatebuilder was assuming existence of - "html" directory in instance home. -- Fixed INSTALL.txt to reflect some changes in the installation and test - procedure. Whatdya know, "setup.py install" does the script install. - There you go... -- Fixed some non-string node ids in cgi_client now that the hyperdb is - strict about such things. - -2001-07-26 - 0.2.0 -Features: -- Major reorganisation of code to allow multiple roundup instances and a - single, site-packages -based installation. Also allows multiple database - back-ends. -- Moved the bin/ proggies into the top dir, so that it all works - out-of-the-box -- Added the "classic" template - a direct implementation of the Roundup - spec. Well, as close as we're going to get, anyway. -- Added an issue priority of support to "extended" -- Added command-line arg handling to roundup-server so it's more useful - out-of-the-box. -- Added distutils-style installation of "lib" files. -- Added some unit tests. - -Fixes: -- Fixed bug in re generation in the filter -- Fixed handling of None String property in grouped list headings -- Fixed adding new issue with no change note -- Fixed values in text input fields which contained quotes (") are now - quoted. -- Fixed a bug in the hyperdb filter - wrong variable names in the error - message. - -2001-07-19 - 0.1.3 -- Reldate now takes an argument "pretty" - when true, it pretty-prints the - interval generated up to 5 days, then pretty-prints the date of last - activity. The issue index and item now use the pretty format. -- Classes list for admin user in CGI interface. -- Made the view configuration more accessible, neater and more realistic. -- Fixed list view grouping handling grouping by a Multilink or String or Link - value of None or Date, ... (mind you, sorting by Date???) -- Fixed bug in the plain formatter when a Link was None. -- Fixed ordering of list view column headings. -- Fixed list view column heading sort links - and limited the number of - columns to sort by to 2. -- Added searching by glob to StringType filtering - - ^text - search for text at start of fields - text$ - search for text at end of fields - ^text$ - exactly match text in fields - te*xt - search for text matching "te"<any characters>"xt" - te?xt - search for text matching "te"<any one character>"xt" -- Added more fields to the issue.filter and issue.index templates - - -2001-07-18 - 0.1.2 -- Set default index to ?:group=priority&:columns=activity,status,title so - the priority column isn't displayed. -- Thanks Anthony: - - added notes to the README about Python prerequisites - - added check to roundup.py, roundup.cgi, server.py and roundup-mailgw.py - for python 2+ - and made the file itself parseable by 1.5.2 ;) - - python 2.0 didn't have the default args for the time module functions. - - better handling of db directory in initDB -- Sorting on the extra properties defined by roundupdb classes was broken - due to the caching used. May now sort on activity and creation - properties, etc. -- Set the default index to sort on activity - - -2001-07-18 - 0.1.1 -- Initial version release with consent of Roundup spec author, Ka-Ping Yee: - "Amazing! Nice work. I'll watch for the source code on your website." - -2001-07-11 - 0.1.0 -- Needed a bug tracking system. Looked around. Tried to install many - Perl-based systems, to no avail. Got tired of waiting for Roundup to be - released. Had just finished major product project, so needed something - different for a while. Roundup here I come... - -
--- a/MANIFEST.in Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,12 +0,0 @@ -recursive-include roundup *.* home* page* -recursive-include frontends *.* -recursive-include scripts *.* *-* -recursive-include tools *.* -recursive-include cgi-bin *.cgi -recursive-include test *.py *.txt -recursive-include doc *.html *.png *.txt -recursive-exclude roundup *.pyc *.pyo .cvsignore -recursive-exclude frontends *.pyc *.pyo .cvsignore -include run_tests *.txt -exclude BUILD.txt I18N_PROGRESS.txt TODO.txt -
--- a/TODO.txt Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,59 +0,0 @@ -General Roundup project TODO list. Note that some of these are semi-formed -ideas. Those ideas that don't make the cutoff for the next major release are -punted automatically into the subsequent major release TODO. When stuff is -done, it's moved to the CHANGES file. - -======= ========= ============================================================ -State Component Description -======= ========= ============================================================ -pending example meta/parent bug implementation (feature request #506815) -pending example replace the "extended" example with a "help desk" one, and - rename "classic" to "bug tracker" -pending example script for retrieval of "mbox" archive of all messages -pending hyperdb range searching of values (dates in particular). - Filter specifies {property: (comparison function, value)} - comparison functions: lt, le, eq, ge, gt. eq and - [value, value, ...] implies "in" -pending hyperdb migrate "id" property to be Number type -pending tracker split instance.open() into open() and login() -pending mailgw allow commands (feature request #556996) - like "help", "dump issue123" (would send all info about - issue123, including a digest of all messages, but probably - not all files...), "list issue", ... -pending mailgw Allow multiple email addresses at one gw with different - default classes and property values (possibly through - command-line args to the mailgw as invoked in the mail - delivery "aliases" file) eg:: - - roundup: "|roundup-mailgw /instances/dev" - vmbugs: "|roundup-mailgw /instances/dev component=voicemail" - -pending mailgw Identification of users should have a configurable degree of - strictness (ie. turn off username==address matching) -pending project switch to a Roundup instance for Roundup bug/feature tracking -pending security an LDAP user database implementation -pending security authenticate over a secure connection -pending security optionally auth with Basic HTTP auth instead of cookies -pending security use digital signatures in mailgw -pending admin "roundup-admin list" should list all the classnames -pending web I18N -pending web Better message summary display (feature request #520244) -pending web Navigating around the issues (feature request #559149) -pending web Quick help links next to the property labels giving a - description of the property. Combine with help for the actual - form element too, eg. how to use the nosy list edit box. -pending web clicking on a group header should filter for that type of - entry -pending web re-enable auth basic http auth -pending web search "refinement" - pre-fill the search page with the - current search parameters -pending web UNIX init.d script for roundup-server -pending web allow multilink selections to select a "none" element to allow - people with broken browsers to select nothing? -pending web automagically link designators -pending web add checkbox-based removal/addition for multilink entries - (eg "add me"/"remove me" for nosy list) - -bug docs need to mention somewhere how sorting works -======= ========= ============================================================= -
--- a/cgi-bin/roundup.cgi Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,213 +0,0 @@ -#!/usr/bin/env python -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: roundup.cgi,v 1.33 2002-09-23 00:50:32 richard Exp $ - -# python version check -from roundup import version_check -from roundup.i18n import _ -import sys - -# -## Configuration -# - -# Configuration can also be provided through the OS environment (or via -# the Apache "SetEnv" configuration directive). If the variables -# documented below are set, they _override_ any configuation defaults -# given in this file. - -# TRACKER_HOMES is a list of instances, in the form -# "NAME=DIR<sep>NAME2=DIR2<sep>...", where <sep> is the directory path -# separator (";" on Windows, ":" on Unix). - -# Make sure the NAME part doesn't include any url-unsafe characters like -# spaces, as these confuse the cookie handling in browsers like IE. - -# ROUNDUP_LOG is the name of the logfile; if it's empty or does not exist, -# logging is turned off (unless you changed the default below). - -# ROUNDUP_DEBUG is a debug level, currently only 0 (OFF) and 1 (ON) are -# used in the code. Higher numbers means more debugging output. - -# This indicates where the Roundup instance lives -TRACKER_HOMES = { - 'demo': '/var/roundup/instances/demo', -} - -# Where to log debugging information to. Use an instance of DevNull if you -# don't want to log anywhere. -class DevNull: - def write(self, info): - pass - def close(self): - pass -#LOG = open('/var/log/roundup.cgi.log', 'a') -LOG = DevNull() - -# -## end configuration -# - - -# -# Set up the error handler -# -try: - import traceback, StringIO, cgi - from roundup.cgi import cgitb -except: - print "Content-Type: text/plain\n" - print _("Failed to import cgitb!\n\n") - s = StringIO.StringIO() - traceback.print_exc(None, s) - print s.getvalue() - - -# -# Check environment for config items -# -def checkconfig(): - import os, string - global TRACKER_HOMES, LOG - - # see if there's an environment var. ROUNDUP_INSTANCE_HOMES is the - # old name for it. - if os.environ.has_key('ROUNDUP_INSTANCE_HOMES'): - homes = os.environ.get('ROUNDUP_INSTANCE_HOMES') - else: - homes = os.environ.get('TRACKER_HOMES', '') - if homes: - TRACKER_HOMES = {} - for home in string.split(homes, os.pathsep): - try: - name, dir = string.split(home, '=', 1) - except ValueError: - # ignore invalid definitions - continue - if name and dir: - TRACKER_HOMES[name] = dir - - logname = os.environ.get('ROUNDUP_LOG', '') - if logname: - LOG = open(logname, 'a') - - # ROUNDUP_DEBUG is checked directly in "roundup.cgi.client" - - -# -# Provide interface to CGI HTTP response handling -# -class RequestWrapper: - '''Used to make the CGI server look like a BaseHTTPRequestHandler - ''' - def __init__(self, wfile): - self.wfile = wfile - def write(self, data): - self.wfile.write(data) - def send_response(self, code): - self.write('Status: %s\r\n'%code) - def send_header(self, keyword, value): - self.write("%s: %s\r\n" % (keyword, value)) - def end_headers(self): - self.write("\r\n") - -# -# Main CGI handler -# -def main(out, err): - import os, string - import roundup.instance - path = string.split(os.environ.get('PATH_INFO', '/'), '/') - request = RequestWrapper(out) - request.path = os.environ.get('PATH_INFO', '/') - instance = path[1] - os.environ['TRACKER_NAME'] = instance - os.environ['PATH_INFO'] = string.join(path[2:], '/') - if TRACKER_HOMES.has_key(instance): - # redirect if we need a trailing '/' - if len(path) == 2: - request.send_response(301) - # redirect - if os.environ.get('HTTPS', '') == 'on': - protocol = 'https' - else: - protocol = 'http' - absolute_url = '%s://%s%s/'%(protocol, os.environ['HTTP_HOST'], - os.environ['REQUEST_URI']) - request.send_header('Location', absolute_url) - request.end_headers() - out.write('Moved Permanently') - else: - instance_home = TRACKER_HOMES[instance] - instance = roundup.instance.open(instance_home) - from roundup.cgi.client import Unauthorised, NotFound - client = instance.Client(instance, request, os.environ) - try: - client.main() - except Unauthorised: - request.send_response(403) - request.send_header('Content-Type', 'text/html') - request.end_headers() - out.write('Unauthorised') - except NotFound: - request.send_response(404) - request.send_header('Content-Type', 'text/html') - request.end_headers() - out.write('Not found: %s'%client.path) - - else: - import urllib - request.send_response(200) - request.send_header('Content-Type', 'text/html') - request.end_headers() - w = request.write - w(_('<html><head><title>Roundup instances index</title></head>\n')) - w(_('<body><h1>Roundup instances index</h1><ol>\n')) - homes = TRACKER_HOMES.keys() - homes.sort() - for instance in homes: - w(_('<li><a href="%(instance_url)s/index">%(instance_name)s</a>\n')%{ - 'instance_url': os.environ['SCRIPT_NAME']+'/'+urllib.quote(instance), - 'instance_name': cgi.escape(instance)}) - w(_('</ol></body></html>')) - -# -# Now do the actual CGI handling -# -out, err = sys.stdout, sys.stderr -try: - # force input/output to binary (important for file up/downloads) - if sys.platform == "win32": - import os, msvcrt - msvcrt.setmode(sys.stdin.fileno(), os.O_BINARY) - msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY) - checkconfig() - sys.stdout = sys.stderr = LOG - main(out, err) -except SystemExit: - pass -except: - sys.stdout, sys.stderr = out, err - out.write('Content-Type: text/html\n\n') - cgitb.handler() -sys.stdout.flush() -sys.stdout, sys.stderr = out, err -LOG.close() - -# vim: set filetype=python ts=4 sw=4 et si
--- a/doc/.cvsignore Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,15 +0,0 @@ -announcement.html -customizing.html -developers.html -getting_started.html -implementation.html -index.html -installation.html -user_guide.html -FAQ.html -security.html -features.html -upgrading.html -glossary.html -design.html -maintenance.html
--- a/doc/FAQ.txt Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,48 +0,0 @@ -=========== -Roundup FAQ -=========== - -:Version: $Revision: 1.10 $ - -NOTE: This is just a grabbag, most of this should go into documentation. - -.. contents:: - - -Changing HTML layout --------------------- - -Installation ------------- - -Living without a mailserver. -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Remove the nosy reactor, means delete the tracker file -'detectors/nosyreactor.py'. - - -Rights issues (MISSING) -~~~~~~~~~~~~~~~~~~~~~~~ - -Different jobs run under different users. - -* Standalone roundup-server is started by whome ? - -* Running cgi under apache. - -* roundup-mailgw called via .forward from MTA, or running a cron job - fetching via pop. - -see Troubleshooting_. - - -Troubleshooting ---------------- - ------------------ - -Back to `Table of Contents`_ - -.. _`Table of Contents`: index.html -
--- a/doc/Makefile Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,17 +0,0 @@ -PYTHON = /usr/bin/python2 -STXTOHTML = -c "from docutils.core import publish;publish(writer_name='html')" - -SOURCE = announcement.txt customizing.txt developers.txt FAQ.txt features.txt \ - getting_started.txt glossary.txt implementation.txt index.txt \ - installation.txt security.txt upgrading.txt user_guide.txt \ - maintenance.txt - -COMPILED := $(SOURCE:.txt=.html) - -all: ${COMPILED} - -%.html: %.txt - ${PYTHON} ${STXTOHTML} --report=warning -d $< $@ - -clean: - rm -f ${COMPILED}
--- a/doc/announcement.txt Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,110 +0,0 @@ -================================================================= -SC-Track Roundup 0.5 final pre-release - an issue tracking system -================================================================= - -Note: This is the final pre-release of the newest version of Roundup. It is - strongly recommended that you maintain your existing 0.4 installation if - you have one, and run 0.5 on a copy of the database. If you are - upgrading from 0.4, you must read doc/upgrading.txt! - -Roundup requires python 2.1.1 for correct operation. Support for dumbdbm -requires python 2.1.2 or 2.2. 2.1.3 and 2.2.1 are recommended. - -This beta release fixes the following specific problems: - -- fixes to import/export -- password edit now has a confirmation field -- cleanups and fixes to the shipped classic template -- new backend for sqlite (and it rocks :) -- many performance improvements in dbm and sql backends -- cgi.client base URL is now obtained from the config TRACKER_WEB (as a result - request.url has gone away - there's too much magic in trying to figure - what it should be) -- cgi-bin script redirects to https now if the request was https -- FileClass "content" property wasn't being returned by getprops() in most - backends -- we now verify instance attributes on instance open and throw a useful error - if they're not all there -- sf bug 611217 ] menu() has problems when labelprop==None -- verify contents of tracker module when the tracker is opened -- fixes to value parsing from edit forms -- mailgw was missing an "import sys" (!) -- setup now installs scripts with python -O flag, doubling performance in some - cases (there's a lot of __debug__ use) -- added getItem to HTMLClass so you can access arbitrary items in templates -- replaced the content() callback ickiness with Page Template macro usage -- changed the default CSS style to be less offensive to some ;) -- better handling of Page Template compilation errors -- sf bug 614188 ] Exception in mailgw.py -- sf bug 613310 ] traceback on onexistant items -- sf bug 613291 ] typos in nosy list -- handle stupid mailers that QUOTE their Re; 'Re: "[issue1] bla blah"' -- giving a user a Role that doesn't exist doesn't break stuff any more -- revamped user guide, customisation guide, added basic maintenance guide -- merge Zope Collector #580 fix from ZPT CVS trunk -- added the "minimal" template - -A lot has been done since 0.4: - -- new backend for metakit (thanks Gordon McMillan) -- new backend for sqlite -- new backend for gadfly (it's as done as it's going to get) -- further split the dbm backends from the core code, allowing easier - non-dict-like backends (eg metakit, RDB) -- added Boolean and Number types -- fixed the journal bloat -- full-text search may also search certain String properties -- entire database export and import (incl files) -- implemented and used the new access control mechanisms (Permissions, Roles) -- switched templating to use Zope's PageTemplates giving much more flexibility -- revamped look and feel in web interface including cleaned up CSS usage -- re-worked cgi interface to abstract out the explicit "issue" interface -- switched to sessions for web authentication -- saving of named search queries -- updated design document for new access controls -- updated customisation document, including more examples -- added maintenance guide -- better mailgw help message (feature request #558562) -- we handle "not found", access and item page render errors better -- fixed double-submit by having new-item-submit redirect at end -- roundup-server may be a daemon now (fork, logfile, pidfile) -- renamed "instance" to "tracker" everywhere, and "node" to "item" in most - places -- many more bug fixes, cleanups and minor improvements - -Source and documentation is available at the website: - http://roundup.sourceforge.net/ -Release Info (via download page): - http://sourceforge.net/projects/roundup -Mailing lists - the place to ask questions: - http://sourceforge.net/mail/?group_id=31577 - - -About Roundup -============= - -Roundup is a simple-to-use and -install issue-tracking system with -command-line, web and e-mail interfaces. It is based on the winning design -from Ka-Ping Yee in the Software Carpentry "Track" design competition. - -Note: Ping is not responsible for this project. The contact for this project -is richard@users.sourceforge.net. - -Roundup manages a number of issues (with flexible properties such as -"description", "priority", and so on) and provides the ability to: - -(a) submit new issues, -(b) find and edit existing issues, and -(c) discuss issues with other participants. - -The system will facilitate communication among the participants by managing -discussions and notifying interested parties when issues are edited. One of -the major design goals for Roundup that it be simple to get going. Roundup -is therefore usable "out of the box" with any python 2.1+ installation. It -doesn't even need to be "installed" to be operational, though a -disutils-based install script is provided. - -It comes with two issue tracker templates (a classic bug/feature tracker and -a minimal skeleton) and six database back-ends (anydbm, bsddb, bsddb3, sqlite, -metakit and gadfly). -
--- a/doc/customizing.txt Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,2072 +0,0 @@ -=================== -Customising Roundup -=================== - -:Version: $Revision: 1.47 $ - -.. This document borrows from the ZopeBook section on ZPT. The original is at: - http://www.zope.org/Documentation/Books/ZopeBook/current/ZPT.stx - -.. contents:: - :depth: 1 - -What You Can Do -=============== - -Customisation of Roundup can take one of five forms: - -1. `tracker configuration`_ file changes -2. database, or `tracker schema`_ changes -3. "definition" class `database content`_ changes -4. behavioural changes, through detectors_ -5. `access controls`_ - -The third case is special because it takes two distinctly different forms -depending upon whether the tracker has been initialised or not. The other two -may be done at any time, before or after tracker initialisation. Yes, this -includes adding or removing properties from classes. - - -Trackers in a Nutshell -====================== - -Trackers have the following structure: - -=================== ======================================================== -Tracker File Description -=================== ======================================================== -config.py Holds the basic `tracker configuration`_ -dbinit.py Holds the `tracker schema`_ -interfaces.py Defines the Web and E-Mail interfaces for the tracker -select_db.py Selects the database back-end for the tracker -db/ Holds the tracker's database -db/files/ Holds the tracker's upload files and messages -detectors/ Auditors and reactors for this tracker -html/ Web interface templates, images and style sheets -=================== ======================================================== - -Tracker Configuration -===================== - -The config.py located in your tracker home contains the basic -configuration for the web and e-mail components of roundup's interfaces. This -file is a Python module. The configuration variables available are: - -**TRACKER_HOME** - ``os.path.split(__file__)[0]`` - The tracker home directory. The above default code will automatically - determine the tracker home for you. - -**MAILHOST** - ``'localhost'`` - The SMTP mail host that roundup will use to send e-mail. - -**MAIL_DOMAIN** - ``'your.tracker.email.domain.example'`` - The domain name used for email addresses. - -**DATABASE** - ``os.path.join(TRACKER_HOME, 'db')`` - This is the directory that the database is going to be stored in. By default - it is in the tracker home. - -**TEMPLATES** - ``os.path.join(TRACKER_HOME, 'html')`` - This is the directory that the HTML templates reside in. By default they are - in the tracker home. - -**TRACKER_NAME** - ``'Roundup issue tracker'`` - A descriptive name for your roundup tracker. This is sent out in e-mails and - appears in the heading of CGI pages. - -**TRACKER_EMAIL** - ``'issue_tracker@%s'%MAIL_DOMAIN`` - The email address that e-mail sent to roundup should go to. Think of it as the - tracker's personal e-mail address. - -**TRACKER_WEB** - ``'http://your.tracker.url.example/'`` - The web address that the tracker is viewable at. This will be included in - information sent to users of the tracker. - -**ADMIN_EMAIL** - ``'roundup-admin@%s'%MAIL_DOMAIN`` - The email address that roundup will complain to if it runs into trouble. - -**MESSAGES_TO_AUTHOR** - ``'yes'`` or``'no'`` - Send nosy messages to the author of the message. - -**ADD_AUTHOR_TO_NOSY** - ``'new'``, ``'yes'`` or ``'no'`` - Does the author of a message get placed on the nosy list automatically? - If ``'new'`` is used, then the author will only be added when a message - creates a new issue. If ``'yes'``, then the author will be added on followups - too. If ``'no'``, they're never added to the nosy. - -**ADD_RECIPIENTS_TO_NOSY** - ``'new'``, ``'yes'`` or ``'no'`` - Do the recipients (To:, Cc:) of a message get placed on the nosy list? - If ``'new'`` is used, then the recipients will only be added when a message - creates a new issue. If ``'yes'``, then the recipients will be added on - followups too. If ``'no'``, they're never added to the nosy. - -**EMAIL_SIGNATURE_POSITION** - ``'top'``, ``'bottom'`` or ``'none'`` - Where to place the email signature in messages that Roundup generates. - -**EMAIL_KEEP_QUOTED_TEXT** - ``'yes'`` or ``'no'`` - Keep email citations. Citations are the part of e-mail which the sender has - quoted in their reply to previous e-mail. - -**EMAIL_LEAVE_BODY_UNCHANGED** - ``'no'`` - Preserve the email body as is. Enabiling this will cause the entire message - body to be stored, including all citations and signatures. It should be - either ``'yes'`` or ``'no'``. - -**MAIL_DEFAULT_CLASS** - ``'issue'`` or ``''`` - Default class to use in the mailgw if one isn't supplied in email - subjects. To disable, comment out the variable below or leave it blank. - -The default config.py is given below - as you -can see, the MAIL_DOMAIN must be edited before any interaction with the -tracker is attempted.:: - - # roundup home is this package's directory - TRACKER_HOME=os.path.split(__file__)[0] - - # The SMTP mail host that roundup will use to send mail - MAILHOST = 'localhost' - - # The domain name used for email addresses. - MAIL_DOMAIN = 'your.tracker.email.domain.example' - - # This is the directory that the database is going to be stored in - DATABASE = os.path.join(TRACKER_HOME, 'db') - - # This is the directory that the HTML templates reside in - TEMPLATES = os.path.join(TRACKER_HOME, 'html') - - # A descriptive name for your roundup tracker - TRACKER_NAME = 'Roundup issue tracker' - - # The email address that mail to roundup should go to - TRACKER_EMAIL = 'issue_tracker@%s'%MAIL_DOMAIN - - # The web address that the tracker is viewable at - TRACKER_WEB = 'http://your.tracker.url.example/' - - # The email address that roundup will complain to if it runs into trouble - ADMIN_EMAIL = 'roundup-admin@%s'%MAIL_DOMAIN - - # Send nosy messages to the author of the message - MESSAGES_TO_AUTHOR = 'no' # either 'yes' or 'no' - - # Does the author of a message get placed on the nosy list automatically? - # If 'new' is used, then the author will only be added when a message - # creates a new issue. If 'yes', then the author will be added on followups - # too. If 'no', they're never added to the nosy. - ADD_AUTHOR_TO_NOSY = 'new' # one of 'yes', 'no', 'new' - - # Do the recipients (To:, Cc:) of a message get placed on the nosy list? - # If 'new' is used, then the recipients will only be added when a message - # creates a new issue. If 'yes', then the recipients will be added on followups - # too. If 'no', they're never added to the nosy. - ADD_RECIPIENTS_TO_NOSY = 'new' # either 'yes', 'no', 'new' - - # Where to place the email signature - EMAIL_SIGNATURE_POSITION = 'bottom' # one of 'top', 'bottom', 'none' - - # Keep email citations - EMAIL_KEEP_QUOTED_TEXT = 'no' # either 'yes' or 'no' - - # Preserve the email body as is - EMAIL_LEAVE_BODY_UNCHANGED = 'no' # either 'yes' or 'no' - - # Default class to use in the mailgw if one isn't supplied in email - # subjects. To disable, comment out the variable below or leave it blank. - # Examples: - MAIL_DEFAULT_CLASS = 'issue' # use "issue" class by default - #MAIL_DEFAULT_CLASS = '' # disable (or just comment the var out) - -Tracker Schema -============== - -Note: if you modify the schema, you'll most likely need to edit the - `web interface`_ HTML template files and `detectors`_ to reflect - your changes. - -A tracker schema defines what data is stored in the tracker's database. -The schemas shipped with Roundup turn it into a typical software bug tracker -or help desk. - -XXX make sure we ship the help desk - -Schemas are defined using Python code in the ``dbinit.py`` module of your -tracker. The "classic" schema looks like this:: - - pri = Class(db, "priority", name=String(), order=String()) - pri.setkey("name") - pri.create(name="critical", order="1") - pri.create(name="urgent", order="2") - pri.create(name="bug", order="3") - pri.create(name="feature", order="4") - pri.create(name="wish", order="5") - - stat = Class(db, "status", name=String(), order=String()) - stat.setkey("name") - stat.create(name="unread", order="1") - stat.create(name="deferred", order="2") - stat.create(name="chatting", order="3") - stat.create(name="need-eg", order="4") - stat.create(name="in-progress", order="5") - stat.create(name="testing", order="6") - stat.create(name="done-cbb", order="7") - stat.create(name="resolved", order="8") - - keyword = Class(db, "keyword", name=String()) - keyword.setkey("name") - - user = Class(db, "user", username=String(), password=String(), - address=String(), realname=String(), phone=String(), - organisation=String()) - user.setkey("username") - user.create(username="admin", password=adminpw, - address=config.ADMIN_EMAIL) - - msg = FileClass(db, "msg", author=Link("user"), recipients=Multilink - ("user"), date=Date(), summary=String(), files=Multilink("file")) - - file = FileClass(db, "file", name=String(), type=String()) - - issue = IssueClass(db, "issue", assignedto=Link("user"), - topic=Multilink("keyword"), priority=Link("priority"), status=Link - ("status")) - issue.setkey('title') - -XXX security definitions - -Classes and Properties - creating a new information store ---------------------------------------------------------- - -In the tracker above, we've defined 7 classes of information: - - priority - Defines the possible levels of urgency for issues. - - status - Defines the possible states of processing the issue may be in. - - keyword - Initially empty, will hold keywords useful for searching issues. - - user - Initially holding the "admin" user, will eventually have an entry for all - users using roundup. - - msg - Initially empty, will all e-mail messages sent to or generated by - roundup. - - file - Initially empty, will all files attached to issues. - - issue - Initially empty, this is where the issue information is stored. - -We define the "priority" and "status" classes to allow two things: reduction in -the amount of information stored on the issue and more powerful, accurate -searching of issues by priority and status. By only requiring a link on the -issue (which is stored as a single number) we reduce the chance that someone -mis-types a priority or status - or simply makes a new one up. - -Class and Items -~~~~~~~~~~~~~~~ - -A Class defines a particular class (or type) of data that will be stored in the -database. A class comprises one or more properties, which given the information -about the class items. -The actual data entered into the database, using class.create() are called -items. They have a special immutable property called id. We sometimes refer to -this as the itemid. - -Properties -~~~~~~~~~~ - -A Class is comprised of one or more properties of the following types: - -* String properties are for storing arbitrary-length strings. -* Password properties are for storing encoded arbitrary-length strings. The - default encoding is defined on the roundup.password.Password class. -* Date properties store date-and-time stamps. Their values are Timestamp - objects. -* Number properties store numeric values. -* Boolean properties store on/off, yes/no, true/false values. -* A Link property refers to a single other item selected from a specified - class. The class is part of the property; the value is an integer, the id - of the chosen item. -* A Multilink property refers to possibly many items in a specified class. - The value is a list of integers. - -FileClass -~~~~~~~~~ - -FileClasses save their "content" attribute off in a separate file from the rest -of the database. This reduces the number of large entries in the database, -which generally makes databases more efficient, and also allows us to use -command-line tools to operate on the files. They are stored in the files sub- -directory of the db directory in your tracker. - -IssueClass -~~~~~~~~~~ - -IssueClasses automatically include the "messages", "files", "nosy", and -"superseder" properties. -The messages and files properties list the links to the messages and files -related to the issue. The nosy property is a list of links to users who wish to -be informed of changes to the issue - they get "CC'ed" e-mails when messages -are sent to or generated by the issue. The nosy reactor (in the detectors -directory) handles this action. The superceder link indicates an issue which -has superceded this one. -They also have the dynamically generated "creation", "activity" and "creator" -properties. -The value of the "creation" property is the date when an item was created, and -the value of the "activity" property is the date when any property on the item -was last edited (equivalently, these are the dates on the first and last -records in the item's journal). The "creator" property holds a link to the user -that created the issue. - -setkey(property) -~~~~~~~~~~~~~~~~ - -Select a String property of the class to be the key property. The key property -muse be unique, and allows references to the items in the class by the content -of the key property. That is, we can refer to users by their username, e.g. -let's say that there's an issue in roundup, issue 23. There's also a user, -richard who happens to be user 2. To assign an issue to him, we could do either -of:: - - roundup-admin set issue assignedto=2 - -or:: - - roundup-admin set issue assignedto=richard - -Note, the same thing can be done in the web and e-mail interfaces. - -create(information) -~~~~~~~~~~~~~~~~~~~ - -Create an item in the database. This is generally used to create items in the -"definitional" classes like "priority" and "status". - - -Examples of adding to your schema ---------------------------------- - -TODO - - -Detectors - adding behaviour to your tracker -============================================ -.. _detectors: - -Detectors are initialised every time you open your tracker database, so you're -free to add and remove them any time, even after the database is initliased -via the "roundup-admin initalise" command. - -The detectors in your tracker fire before (*auditors*) and after (*reactors*) -changes to the contents of your database. They are Python modules that sit in -your tracker's ``detectors`` directory. You will have some installed by -default - have a look. You can write new detectors or modify the existing -ones. The existing detectors installed for you are: - -**nosyreaction.py** - This provides the automatic nosy list maintenance and email sending. The nosy - reactor (``nosyreaction``) fires when new messages are added to issues. - The nosy auditor (``updatenosy``) fires when issues are changed and figures - what changes need to be made to the nosy list (like adding new authors etc) -**statusauditor.py** - This provides the ``chatty`` auditor which changes the issue status from - ``unread`` or ``closed`` to ``chatting`` if new messages appear. It also - provides the ``presetunread`` auditor which pre-sets the status to - ``unread`` on new items if the status isn't explicitly defined. - -See the detectors section in the `design document`__ for details of the -interface for detectors. - -__ design.html - -Sample additional detectors that have been found useful will appear in the -``detectors`` directory of the Roundup distribution: - -**newissuecopy.py** - This detector sends an email to a team address whenever a new issue is - created. The address is hard-coded into the detector, so edit it before you - use it (look for the text 'team@team.host') or you'll get email errors! - - The detector code:: - - from roundup import roundupdb - - def newissuecopy(db, cl, nodeid, oldvalues): - ''' Copy a message about new issues to a team address. - ''' - # so use all the messages in the create - change_note = cl.generateCreateNote(nodeid) - - # send a copy to the nosy list - for msgid in cl.get(nodeid, 'messages'): - try: - # note: last arg must be a list - cl.send_message(nodeid, msgid, change_note, ['team@team.host']) - except roundupdb.MessageSendError, message: - raise roundupdb.DetectorError, message - - def init(db): - db.issue.react('create', newissuecopy) - - -Database Content -================ - -Note: if you modify the content of definitional classes, you'll most likely - need to edit the tracker `detectors`_ to reflect your changes. - -Customisation of the special "definitional" classes (eg. status, priority, -resolution, ...) may be done either before or after the tracker is -initialised. The actual method of doing so is completely different in each -case though, so be careful to use the right one. - -**Changing content before tracker initialisation** - Edit the dbinit module in your tracker to alter the items created in using - the create() methods. - -**Changing content after tracker initialisation** - Use the roundup-admin interface's create, set and retire methods to add, - alter or remove items from the classes in question. - -XXX example - - -Web Interface -============= - -.. contents:: - :local: - :depth: 1 - -The web is provided by the roundup.cgi.client module and is used by -roundup.cgi, roundup-server and ZRoundup. -In all cases, we determine which tracker is being accessed -(the first part of the URL path inside the scope of the CGI handler) and pass -control on to the tracker interfaces.Client class - which uses the Client class -from roundup.cgi.client - which handles the rest of -the access through its main() method. This means that you can do pretty much -anything you want as a web interface to your tracker. - -Repurcussions of changing the tracker schema ---------------------------------------------- - -If you choose to change the `tracker schema`_ you will need to ensure the web -interface knows about it: - -1. Index, item and search pages for the relevant classes may need to have - properties added or removed, -2. The "page" template may require links to be changed, as might the "home" - page's content arguments. - -How requests are processed --------------------------- - -The basic processing of a web request proceeds as follows: - -1. figure out who we are, defaulting to the "anonymous" user -2. figure out what the request is for - we call this the "context" -3. handle any requested action (item edit, search, ...) -4. render the template requested by the context, resulting in HTML output - -In some situations, exceptions occur: - -- HTTP Redirect (generally raised by an action) -- SendFile (generally raised by determine_context) - here we serve up a FileClass "content" property -- SendStaticFile (generally raised by determine_context) - here we serve up a file from the tracker "html" directory -- Unauthorised (generally raised by an action) - here the action is cancelled, the request is rendered and an error - message is displayed indicating that permission was not - granted for the action to take place -- NotFound (raised wherever it needs to be) - this exception percolates up to the CGI interface that called the client - -Determining web context ------------------------ - -To determine the "context" of a request, we look at the URL and the special -request variable ``:template``. The URL path after the tracker identifier -is examined. Typical URL paths look like: - -1. ``/tracker/issue`` -2. ``/tracker/issue1`` -3. ``/tracker/_file/style.css`` -4. ``/cgi-bin/roundup.cgi/tracker/file1`` -5. ``/cgi-bin/roundup.cgi/tracker/file1/kitten.png`` - -where the "tracker identifier" is "tracker" in the above cases. That means -we're looking at "issue", "issue1", "_file/style.css", "file1" and -"file1/kitten.png" in the cases above. The path is generally only one -entry long - longer paths are handled differently. - -a. if there is no path, then we are in the "home" context. -b. if the path starts with "_file" (as in example 3, - "/tracker/_file/style.css"), then the additional path entry, - "style.css" specifies the filename of a static file we're to serve up - from the tracker "html" directory. Raises a SendStaticFile - exception. -c. if there is something in the path (as in example 1, "issue"), it identifies - the tracker class we're to display. -d. if the path is an item designator (as in examples 2 and 4, "issue1" and - "file1"), then we're to display a specific item. -e. if the path starts with an item designator and is longer than - one entry (as in example 5, "file1/kitten.png"), then we're assumed - to be handling an item of a - FileClass, and the extra path information gives the filename - that the client is going to label the download with (ie - "file1/kitten.png" is nicer to download than "file1"). This - raises a SendFile exception. - -Both b. and e. stop before we bother to -determine the template we're going to use. That's because they -don't actually use templates. - -The template used is specified by the ``:template`` CGI variable, -which defaults to: - -- only classname suplied: "index" -- full item designator supplied: "item" - - -Performing actions in web requests ----------------------------------- - -When a user requests a web page, they may optionally also request for an -action to take place. As described in `how requests are processed`_, the -action is performed before the requested page is generated. Actions are -triggered by using a ``:action`` CGI variable, where the value is one of: - -**login** - Attempt to log a user in. -**logout** - Log the user out - make them "anonymous". -**register** - Attempt to create a new user based on the contents of the form and then log - them in. -**edit** - Perform an edit of an item in the database. There are some special form - elements you may use: - - :link=designator:property and :multilink=designator:property - The value specifies an item designator and the property on that - item to add _this_ item to as a link or multilink. - :note - Create a message and attach it to the current item's - "messages" property. - :file - Create a file and attach it to the current item's - "files" property. Attach the file to the message created from - the :note if it's supplied. - :required=property,property,... - The named properties are required to be filled in the form. - -**new** - Add a new item to the database. You may use the same special form elements - as in the "edit" action. - -**editCSV** - Performs an edit of all of a class' items in one go. See also the - *class*.csv templating method which generates the CSV data to be edited, and - the "_generic.index" template which uses both of these features. - -**search** - Mangle some of the form variables. - - Set the form ":filter" variable based on the values of the - filter variables - if they're set to anything other than - "dontcare" then add them to :filter. - - Also handle the ":queryname" variable and save off the query to - the user's query list. - -Each of the actions is implemented by a corresponding *actionAction* (where -"action" is the name of the action) method on -the roundup.cgi.Client class, which also happens to be in your tracker as -interfaces.Client. So if you need to define new actions, you may add them -there (see `defining new web actions`_). - -Each action also has a corresponding *actionPermission* (where -"action" is the name of the action) method which determines -whether the action is permissible given the current user. The base permission -checks are: - -**login** - Determine whether the user has permission to log in. - Base behaviour is to check the user has "Web Access". -**logout** - No permission checks are made. -**register** - Determine whether the user has permission to register - Base behaviour is to check the user has "Web Registration". -**edit** - Determine whether the user has permission to edit this item. - Base behaviour is to check the user can edit this class. If we're - editing the "user" class, users are allowed to edit their own - details. Unless it's the "roles" property, which requires the - special Permission "Web Roles". -**new** - Determine whether the user has permission to create (edit) this item. - Base behaviour is to check the user can edit this class. No - additional property checks are made. Additionally, new user items - may be created if the user has the "Web Registration" Permission. -**editCSV** - Determine whether the user has permission to edit this class. - Base behaviour is to check the user can edit this class. -**search** - Determine whether the user has permission to search this class. - Base behaviour is to check the user can view this class. - - -Default templates ------------------ - -Most customisation of the web view can be done by modifying the templates in -the tracker **html** directory. There are several types of files in there: - -**page** - This template usually defines the overall look of your tracker. When you - view an issue, it appears inside this template. When you view an index, it - also appears inside this template. This template defines a macro called - "icing" which is used by almost all other templates as a coating for their - content, using its "content" slot. It will also define the "head_title" - and "body_title" slots to allow setting of the page title. -**home** - the default page displayed when no other page is indicated by the user -**home.classlist** - a special version of the default page that lists the classes in the tracker -**classname.item** - displays an item of the *classname* class -**classname.index** - displays a list of *classname* items -**classname.search** - displays a search page for *classname* items -**_generic.index** - used to display a list of items where there is no *classname*.index available -**_generic.help** - used to display a "class help" page where there is no *classname*.help -**user.register** - a special page just for the user class that renders the registration page -**style.css** - a static file that is served up as-is - -Note: Remember that you can create any template extension you want to, so -if you just want to play around with the templating for new issues, you can -copy the current "issue.item" template to "issue.test", and then access the -test template using the ":template" URL argument:: - - http://your.tracker.example/tracker/issue?:template=test - -and it won't affect your users using the "issue.item" template. - - -How the templates work ----------------------- - -Basic Templating Actions -~~~~~~~~~~~~~~~~~~~~~~~~ - -Roundup's templates consist of special attributes on your template tags. -These attributes form the Template Attribute Language, or TAL. The basic tag -commands are: - -**tal:define="variable expression; variable expression; ..."** - Define a new variable that is local to this tag and its contents. For - example:: - - <html tal:define="title request/description"> - <head><title tal:content="title"></title></head> - </html> - - In the example, the variable "title" is defined as being the result of the - expression "request/description". The tal:content command inside the <html> - tag may then use the "title" variable. - -**tal:condition="expression"** - Only keep this tag and its contents if the expression is true. For example:: - - <p tal:condition="python:request.user.hasPermission('View', 'issue')"> - Display some issue information. - </p> - - In the example, the <p> tag and its contents are only displayed if the - user has the View permission for issues. We consider the number zero, a - blank string, an empty list, and the built-in variable nothing to be false - values. Nearly every other value is true, including non-zero numbers, and - strings with anything in them (even spaces!). - -**tal:repeat="variable expression"** - Repeat this tag and its contents for each element of the sequence that the - expression returns, defining a new local variable and a special "repeat" - variable for each element. For example:: - - <tr tal:repeat="u user/list"> - <td tal:content="u/id"></td> - <td tal:content="u/username"></td> - <td tal:content="u/realname"></td> - </tr> - - The example would iterate over the sequence of users returned by - "user/list" and define the local variable "u" for each entry. - -**tal:replace="expression"** - Replace this tag with the result of the expression. For example:: - - <span tal:replace="request/user/realname"></span> - - The example would replace the <span> tag and its contents with the user's - realname. If the user's realname was "Bruce" then the resultant output - would be "Bruce". - -**tal:content="expression"** - Replace the contents of this tag with the result of the expression. For - example:: - - <span tal:content="request/user/realname">user's name appears here</span> - - The example would replace the contents of the <span> tag with the user's - realname. If the user's realname was "Bruce" then the resultant output - would be "<span>Bruce</span>". - -**tal:attributes="attribute expression; attribute expression; ..."** - Set attributes on this tag to the results of expressions. For example:: - - <a tal:attributes="href string:user${request/user/id}">My Details</a> - - In the example, the "href" attribute of the <a> tag is set to the value of - the "string:user${request/user/id}" expression, which will be something - like "user123". - -**tal:omit-tag="expression"** - Remove this tag (but not its contents) if the expression is true. For - example:: - - <span tal:omit-tag="python:1">Hello, world!</span> - - would result in output of:: - - Hello, world! - -Note that the commands on a given tag are evaulated in the order above, so -*define* comes before *condition*, and so on. - -Additionally, a tag is defined, tal:block, which is removed from output. Its -content is not, but the tag itself is (so don't go using any tal:attributes -commands on it). This is useful for making arbitrary blocks of HTML -conditional or repeatable (very handy for repeating multiple table rows, -which would othewise require an illegal tag placement to effect the repeat). - - -Templating Expressions -~~~~~~~~~~~~~~~~~~~~~~ - -The expressions you may use in the attibute values may be one of the following -forms: - -**Path Expressions** - eg. ``item/status/checklist`` - These are object attribute / item accesses. Roughly speaking, the path - ``item/status/checklist`` is broken into parts ``item``, ``status`` - and ``checklist``. The ``item`` part is the root of the expression. - We then look for a ``status`` attribute on ``item``, or failing that, a - ``status`` item (as in ``item['status']``). If that - fails, the path expression fails. When we get to the end, the object we're - left with is evaluated to get a string - methods are called, objects are - stringified. Path expressions may have an optional ``path:`` prefix, though - they are the default expression type, so it's not necessary. - - XXX | components of expressions - - XXX "nothing" and "default" - -**String Expressions** - eg. ``string:hello ${user/name}`` - These expressions are simple string interpolations (though they can be just - plain strings with no interpolation if you want. The expression in the - ``${ ... }`` is just a path expression as above. - -**Python Expressions** - eg. ``python: 1+1`` - These expressions give the full power of Python. All the "root level" - variables are available, so ``python:item.status.checklist()`` would be - equivalent to ``item/status/checklist``, assuming that ``checklist`` is - a method. - -Template Macros -~~~~~~~~~~~~~~~ - -Macros are used in Roundup to save us from repeating the same common page -stuctures over and over. The most common (and probably only) macro you'll use -is the "icing" macro defined in the "page" template. - -Macros are generated and used inside your templates using special attributes -similar to the `basic templating actions`_. In this case though, the -attributes belong to the Macro Expansion Template Attribute Language, or -METAL. The macro commands are: - -**metal:define-macro="macro name"** - Define that the tag and its contents are now a macro that may be inserted - into other templates using the *use-macro* command. For example:: - - <html metal:define-macro="page"> - ... - </html> - - defines a macro called "page" using the ``<html>`` tag and its contents. - Once defined, macros are stored on the template they're defined on in the - ``macros`` attribute. You can access them later on through the ``templates`` - variable, eg. the most common ``templates/page/macros/icing`` to access the - "page" macro of the "page" template. - -**metal:use-macro="path expression"** - Use a macro, which is identified by the path expression (see above). This - will replace the current tag with the identified macro contents. For - example:: - - <tal:block metal:use-macro="templates/page/macros/icing"> - ... - </tal:block> - - will replace the tag and its contents with the "page" macro of the "page" - template. - -**metal:define-slot="slot name"** and **metal:fill-slot="slot name"** - To define *dynamic* parts of the macro, you define "slots" which may be - filled when the macro is used with a *use-macro* command. For example, the - ``templates/page/macros/icing`` macro defines a slot like so:: - - <title metal:define-slot="head_title">title goes here</title> - - In your *use-macro* command, you may now use a *fill-slot* command like - this:: - - <title metal:fill-slot="head_title">My Title</title> - - where the tag that fills the slot completely replaces the one defined as - the slot in the macro. - -Note that you may not mix METAL and TAL commands on the same tag, but TAL -commands may be used freely inside METAL-using tags (so your *fill-slots* -tags may have all manner of TAL inside them). - - -Information available to templates ----------------------------------- - -Note: this is implemented by roundup.cgi.templating.RoundupPageTemplate - -The following variables are available to templates. - -**context** - The current context. This is either None, a - `hyperdb class wrapper`_ or a `hyperdb item wrapper`_ -**request** - Includes information about the current request, including: - - the url - - the current index information (``filterspec``, ``filter`` args, - ``properties``, etc) parsed out of the form. - - methods for easy filterspec link generation - - *user*, the current user item as an HTMLItem instance - - *form* - The current CGI form information as a mapping of form argument - name to value -**tracker** - The current tracker -**db** - The current database, through which db.config may be reached. -**templates** - Access to all the tracker templates by name. Used mainly in *use-macro* - commands. -**utils** - This variable makes available some utility functions like batching. -**nothing** - This is a special variable - if an expression evaluates to this, then the - tag (in the case of a tal:replace), its contents (in the case of - tal:content) or some attributes (in the case of tal:attributes) will not - appear in the the output. So for example:: - - <span tal:attributes="class nothing">Hello, World!</span> - - would result in:: - - <span>Hello, World!</span> - -**default** - Also a special variable - if an expression evaluates to this, then the - existing HTML in the template will not be replaced or removed, it will - remain. So:: - - <span tal:replace="default">Hello, World!</span> - - would result in:: - - <span>Hello, World!</span> - -The context variable -~~~~~~~~~~~~~~~~~~~~ - -The *context* variable is one of three things based on the current context -(see `determining web context`_ for how we figure this out): - -1. if we're looking at a "home" page, then it's None -2. if we're looking at a specific hyperdb class, it's a - `hyperdb class wrapper`_. -3. if we're looking at a specific hyperdb item, it's a - `hyperdb item wrapper`_. - -If the context is not None, we can access the properties of the class or item. -The only real difference between cases 2 and 3 above are: - -1. the properties may have a real value behind them, and this will appear if - the property is displayed through ``context/property`` or - ``context/property/field``. -2. the context's "id" property will be a false value in the second case, but - a real, or true value in the third. Thus we can determine whether we're - looking at a real item from the hyperdb by testing "context/id". - -Hyperdb class wrapper -::::::::::::::::::::: - -Note: this is implemented by the roundup.cgi.templating.HTMLClass class. - -This wrapper object provides access to a hyperb class. It is used primarily -in both index view and new item views, but it's also usable anywhere else that -you wish to access information about a class, or the items of a class, when -you don't have a specific item of that class in mind. - -We allow access to properties. There will be no "id" property. The value -accessed through the property will be the current value of the same name from -the CGI form. - -There are several methods available on these wrapper objects: - -=========== ============================================================= -Method Description -=========== ============================================================= -properties return a `hyperdb property wrapper`_ for all of this class' - properties. -list lists all of the active (not retired) items in the class. -csv return the items of this class as a chunk of CSV text. -propnames lists the names of the properties of this class. -filter lists of items from this class, filtered and sorted - by the current *request* filterspec/filter/sort/group args -classhelp display a link to a javascript popup containing this class' - "help" template. -submit generate a submit button (and action hidden element) -renderWith render this class with the given template. -history returns 'New node - no history' :) -is_edit_ok is the user allowed to Edit the current class? -is_view_ok is the user allowed to View the current class? -=========== ============================================================= - -Note that if you have a property of the same name as one of the above methods, -you'll need to access it using a python "item access" expression. For example:: - - python:context['list'] - -will access the "list" property, rather than the list method. - - -Hyperdb item wrapper -:::::::::::::::::::: - -Note: this is implemented by the roundup.cgi.templating.HTMLItem class. - -This wrapper object provides access to a hyperb item. - -We allow access to properties. There will be no "id" property. The value -accessed through the property will be the current value of the same name from -the CGI form. - -There are several methods available on these wrapper objects: - -=============== ============================================================= -Method Description -=============== ============================================================= -submit generate a submit button (and action hidden element) -journal return the journal of the current item (**not implemented**) -history render the journal of the current item as HTML -renderQueryForm specific to the "query" class - render the search form for - the query -hasPermission specific to the "user" class - determine whether the user - has a Permission -is_edit_ok is the user allowed to Edit the current item? -is_view_ok is the user allowed to View the current item? -=============== ============================================================= - - -Note that if you have a property of the same name as one of the above methods, -you'll need to access it using a python "item access" expression. For example:: - - python:context['journal'] - -will access the "journal" property, rather than the journal method. - - -Hyperdb property wrapper -:::::::::::::::::::::::: - -Note: this is implemented by subclasses roundup.cgi.templating.HTMLProperty -class (HTMLStringProperty, HTMLNumberProperty, and so on). - -This wrapper object provides access to a single property of a class. Its -value may be either: - -1. if accessed through a `hyperdb item wrapper`_, then it's a value from the - hyperdb -2. if access through a `hyperdb class wrapper`_, then it's a value from the - CGI form - - -The property wrapper has some useful attributes: - -=============== ============================================================= -Attribute Description -=============== ============================================================= -_name the name of the property -_value the value of the property if any -=============== ============================================================= - -There are several methods available on these wrapper objects: - -=========== ================================================================= -Method Description -=========== ================================================================= -plain render a "plain" representation of the property -field render a form edit field for the property -stext only on String properties - render the value of the - property as StructuredText (requires the StructureText module - to be installed separately) -multiline only on String properties - render a multiline form edit - field for the property -email only on String properties - render the value of the - property as an obscured email address -confirm only on Password properties - render a second form edit field for - the property, used for confirmation that the user typed the - password correctly. Generates a field with name "name:confirm". -reldate only on Date properties - render the interval between the - date and now -pretty only on Interval properties - render the interval in a - pretty format (eg. "yesterday") -menu only on Link and Multilink properties - render a form select - list for this property -reverse only on Multilink properties - produce a list of the linked - items in reverse order -=========== ================================================================= - -The request variable -~~~~~~~~~~~~~~~~~~~~ - -Note: this is implemented by the roundup.cgi.templating.HTMLRequest class. - -The request variable is packed with information about the current request. - -.. taken from roundup.cgi.templating.HTMLRequest docstring - -=========== ================================================================= -Variable Holds -=========== ================================================================= -form the CGI form as a cgi.FieldStorage -env the CGI environment variables -url the current URL path for this request -base the base URL for this tracker -user a HTMLUser instance for this user -classname the current classname (possibly None) -template the current template (suffix, also possibly None) -form the current CGI form variables in a FieldStorage -=========== ================================================================= - -**Index page specific variables (indexing arguments)** - -=========== ================================================================= -Variable Holds -=========== ================================================================= -columns dictionary of the columns to display in an index page -show a convenience access to columns - request/show/colname will - be true if the columns should be displayed, false otherwise -sort index sort column (direction, column name) -group index grouping property (direction, column name) -filter properties to filter the index on -filterspec values to filter the index on -search_text text to perform a full-text search on for an index -=========== ================================================================= - -There are several methods available on the request variable: - -=============== ============================================================= -Method Description -=============== ============================================================= -description render a description of the request - handle for the page - title -indexargs_form render the current index args as form elements -indexargs_url render the current index args as a URL -base_javascript render some javascript that is used by other components of - the templating -batch run the current index args through a filter and return a - list of items (see `hyperdb item wrapper`_, and - `batching`_) -=============== ============================================================= - -The form variable -::::::::::::::::: - -The form variable is a little special because it's actually a python -FieldStorage object. That means that you have two ways to access its -contents. For example, to look up the CGI form value for the variable -"name", use the path expression:: - - request/form/name/value - -or the python expression:: - - python:request.form['name'].value - -Note the "item" access used in the python case, and also note the explicit -"value" attribute we have to access. That's because the form variables are -stored as MiniFieldStorages. If there's more than one "name" value in -the form, then the above will break since ``request/form/name`` is actually a -*list* of MiniFieldStorages. So it's best to know beforehand what you're -dealing with. - - -The db variable -~~~~~~~~~~~~~~~ - -Note: this is implemented by the roundup.cgi.templating.HTMLDatabase class. - -Allows access to all hyperdb classes as attributes of this variable. If you -want access to the "user" class, for example, you would use:: - - db/user - python:db.user - -The access results in a `hyperdb class wrapper`_. - -The templates variable -~~~~~~~~~~~~~~~~~~~~~~ - -Note: this is implemented by the roundup.cgi.templating.Templates class. - -This variable doesn't have any useful methods defined. It supports being -used in expressions to access the templates, and subsequently the template -macros. You may access the templates using the following path expression:: - - templates/name - -or the python expression:: - - templates[name] - -where "name" is the name of the template you wish to access. The template you -get access to has one useful attribute, "macros". To access a specific macro -(called "macro_name"), use the path expression:: - - templates/name/macros/macro_name - -or the python expression:: - - templates[name].macros[macro_name] - - -The utils variable -~~~~~~~~~~~~~~~~~~ - -Note: this is implemented by the roundup.cgi.templating.TemplatingUtils class. - -=============== ============================================================= -Method Description -=============== ============================================================= -Batch return a batch object using the supplied list -=============== ============================================================= - -Batching -:::::::: - -Use Batch to turn a list of items, or item ids of a given class, into a series -of batches. Its usage is:: - - python:util.Batch(sequence, size, start, end=0, orphan=0, overlap=0) - -or, to get the current index batch:: - - request/batch - -The parameters are: - -========= ================================================================== -Parameter Usage -========= ================================================================== -sequence a list of HTMLItems -size how big to make the sequence. -start where to start (0-indexed) in the sequence. -end where to end (0-indexed) in the sequence. -orphan if the next batch would contain less items than this - value, then it is combined with this batch -overlap the number of items shared between adjacent batches -========= ================================================================== - -All of the parameters are assigned as attributes on the batch object. In -addition, it has several more attributes: - -=============== ============================================================ -Attribute Description -=============== ============================================================ -start indicates the start index of the batch. *Note: unlike the - argument, is a 1-based index (I know, lame)* -first indicates the start index of the batch *as a 0-based - index* -length the actual number of elements in the batch -sequence_length the length of the original, unbatched, sequence. -=============== ============================================================ - -And several methods: - -=============== ============================================================ -Method Description -=============== ============================================================ -previous returns a new Batch with the previous batch settings -next returns a new Batch with the next batch settings -propchanged detect if the named property changed on the current item - when compared to the last item -=============== ============================================================ - -An example of batching:: - - <table class="otherinfo"> - <tr><th colspan="4" class="header">Existing Keywords</th></tr> - <tr tal:define="keywords db/keyword/list" - tal:repeat="start python:range(0, len(keywords), 4)"> - <td tal:define="batch python:utils.Batch(keywords, 4, start)" - tal:repeat="keyword batch" tal:content="keyword/name">keyword here</td> - </tr> - </table> - -... which will produce a table with four columns containing the items of the -"keyword" class (well, their "name" anyway). - -Displaying Properties ---------------------- - -Properties appear in the user interface in three contexts: in indices, in -editors, and as search arguments. -For each type of property, there are several display possibilities. -For example, in an index view, a string property may just be -printed as a plain string, but in an editor view, that property may be -displayed in an editable field. - - -Index Views ------------ - -This is one of the class context views. It is also the default view for -classes. The template used is "*classname*.index". - -Index View Specifiers -~~~~~~~~~~~~~~~~~~~~~ - -An index view specifier (URL fragment) looks like this (whitespace has been -added for clarity):: - - /issue?status=unread,in-progress,resolved& - topic=security,ui& - :group=+priority& - :sort==activity& - :filters=status,topic& - :columns=title,status,fixer - -The index view is determined by two parts of the specifier: the layout part and -the filter part. The layout part consists of the query parameters that begin -with colons, and it determines the way that the properties of selected items -are displayed. The filter part consists of all the other query parameters, and -it determines the criteria by which items are selected for display. -The filter part is interactively manipulated with the form widgets displayed in -the filter section. The layout part is interactively manipulated by clicking on -the column headings in the table. - -The filter part selects the union of the sets of items with values matching any -specified Link properties and the intersection of the sets of items with values -matching any specified Multilink properties. - -The example specifies an index of "issue" items. Only items with a "status" of -either "unread" or "in-progres" or "resolved" are displayed, and only items -with "topic" values including both "security" and "ui" are displayed. The items -are grouped by priority, arranged in ascending order; and within groups, sorted -by activity, arranged in descending order. The filter section shows filters for -the "status" and "topic" properties, and the table includes columns for the -"title", "status", and "fixer" properties. - -Filtering of indexes -~~~~~~~~~~~~~~~~~~~~ - -TODO - -Searching Views ---------------- - -This is one of the class context views. The template used is typically -"*classname*.search". - -TODO - -Item Views ----------- - -The basic view of a hyperdb item is provided by the "*classname*.item" -template. It generally has three sections; an "editor", a "spool" and a -"history" section. - - - -Editor Section -~~~~~~~~~~~~~~ - -The editor section is used to manipulate the item - it may be a -static display if the user doesn't have permission to edit the item. - -Here's an example of a basic editor template (this is the default "classic" -template issue item edit form - from the "issue.item" template):: - - <table class="form"> - <tr> - <th nowrap>Title</th> - <td colspan=3 tal:content="structure python:context.title.field(size=60)">title</td> - </tr> - - <tr> - <th nowrap>Priority</th> - <td tal:content="structure context/priority/menu">priority</td> - <th nowrap>Status</th> - <td tal:content="structure context/status/menu">status</td> - </tr> - - <tr> - <th nowrap>Superseder</th> - <td> - <span tal:replace="structure python:context.superseder.field(showid=1, size=20)" /> - <span tal:replace="structure python:db.issue.classhelp('id,title')" /> - <span tal:condition="context/superseder"> - <br>View: <span tal:replace="structure python:context.superseder.link(showid=1)" /> - </span> - </td> - <th nowrap>Nosy List</th> - <td> - <span tal:replace="structure context/nosy/field" /> - <span tal:replace="structure python:db.user.classhelp('username,realname,address,phone')" /> - </td> - </tr> - - <tr> - <th nowrap>Assigned To</th> - <td tal:content="structure context/assignedto/menu"> - assignedto menu - </td> - <td> </td> - <td> </td> - </tr> - - <tr> - <th nowrap>Change Note</th> - <td colspan=3> - <textarea name=":note" wrap="hard" rows="5" cols="60"></textarea> - </td> - </tr> - - <tr> - <th nowrap>File</th> - <td colspan=3><input type="file" name=":file" size="40"></td> - </tr> - - <tr> - <td> </td> - <td colspan=3 tal:content="structure context/submit"> - submit button will go here - </td> - </tr> - </table> - - -When a change is submitted, the system automatically generates a message -describing the changed properties. As shown in the example, the editor -template can use the ":note" and ":file" fields, which are added to the -standard change note message generated by Roundup. - -Spool Section -~~~~~~~~~~~~~ - -The spool section lists related information like the messages and files of -an issue. - -TODO - - -History Section -~~~~~~~~~~~~~~~ - -The final section displayed is the history of the item - its database journal. -This is generally generated with the template:: - - <tal:block tal:replace="structure context/history" /> - -*To be done:* - -*The actual history entries of the item may be accessed for manual templating -through the "journal" method of the item*:: - - <tal:block tal:repeat="entry context/journal"> - a journal entry - </tal:block> - -*where each journal entry is an HTMLJournalEntry.* - -Defining new web actions ------------------------- - -XXX - - -Access Controls -=============== - -A set of Permissions are built in to the security module by default: - -- Edit (everything) -- View (everything) - -The default interfaces define: - -- Web Registration -- Web Access -- Web Roles -- Email Registration -- Email Access - -These are hooked into the default Roles: - -- Admin (Edit everything, View everything, Web Roles) -- User (Web Access, Email Access) -- Anonymous (Web Registration, Email Registration) - -And finally, the "admin" user gets the "Admin" Role, and the "anonymous" user -gets the "Anonymous" assigned when the database is initialised on installation. -The two default schemas then define: - -- Edit issue, View issue (both) -- Edit file, View file (both) -- Edit msg, View msg (both) -- Edit support, View support (extended only) - -and assign those Permissions to the "User" Role. New users are assigned the -Roles defined in the config file as: - -- NEW_WEB_USER_ROLES -- NEW_EMAIL_USER_ROLES - -You may alter the configuration variables to change the Role that new web or -email users get, for example to not give them access to the web interface if -they register through email. - -You may use the ``roundup-admin`` "``security``" command to display the -current Role and Permission configuration in your tracker. - -Adding a new Permission ------------------------ - -When adding a new Permission, you will need to: - -1. add it to your tracker's dbinit so it is created -2. enable it for the Roles that should have it (verify with - "``roundup-admin security``") -3. add it to the relevant HTML interface templates -4. add it to the appropriate xxxPermission methods on in your tracker - interfaces module - - - -Examples -======== - -Adding a new field to the classic schema ----------------------------------------- - -This example shows how to add a new constrained property (ie. a selection of -distinct values) to your tracker. - -Introduction -~~~~~~~~~~~~ - -To make the classic schema of roundup useful as a todo tracking system -for a group of systems administrators, it needed an extra data field -per issue: a category. - -This would let sysads quickly list all todos in their particular -area of interest without having to do complex queries, and without -relying on the spelling capabilities of other sysads (a losing -proposition at best). - -Adding a field to the database -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -This is the easiest part of the change. The category would just be a plain -string, nothing fancy. To change what is in the database you need to add -some lines to the ``open()`` function in ``dbinit.py``:: - - category = Class(db, "category", name=String()) - category.setkey("name") - -Here we are setting up a chunk of the database which we are calling -"category". It contains a string, which we are refering to as "name" for -lack of a more imaginative title. Then we are setting the key of this chunk -of the database to be that "name". This is equivalent to an index for -database types. This also means that there can only be one category with a -given name. - -Adding the above lines allows us to create categories, but they're not tied -to the issues that we are going to be creating. It's just a list of categories -off on its own, which isn't much use. We need to link it in with the issues. -To do that, find the lines in the ``open()`` function in ``dbinit.py`` which -set up the "issue" class, and then add a link to the category:: - - issue = IssueClass(db, "issue", ... , category=Multilink("category"), ... ) - -The Multilink() means that each issue can have many categories. If you were -adding something with a more one to one relationship use Link() instead. - -That is all you need to do to change the schema. The rest of the effort is -fiddling around so you can actually use the new category. - -Setting up security on the new objects -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -By default only the admin user can look at and change objects. This doesn't -suit us, as we want any user to be able to create new categories as -required, and obviously everyone needs to be able to view the categories of -issues for it to be useful. - -We therefore need to change the security of the category objects. This is -also done in the ``open()`` function of ``dbinit.py``. - -There are currently two loops which set up permissions and then assign them -to various roles. Simply add the new "category" to both lists:: - - # new permissions for this schema - for cl in 'issue', 'file', 'msg', 'user', 'category': - db.security.addPermission(name="Edit", klass=cl, - description="User is allowed to edit "+cl) - db.security.addPermission(name="View", klass=cl, - description="User is allowed to access "+cl) - - # Assign the access and edit permissions for issue, file and message - # to regular users now - for cl in 'issue', 'file', 'msg', 'category': - p = db.security.getPermission('View', cl) - db.security.addPermissionToRole('User', p) - p = db.security.getPermission('Edit', cl) - db.security.addPermissionToRole('User', p) - -So you are in effect doing the following:: - - db.security.addPermission(name="Edit", klass='category', - description="User is allowed to edit "+'category') - db.security.addPermission(name="View", klass='category', - description="User is allowed to access "+'category') - -which is creating two permission types; that of editing and viewing -"category" objects respectively. Then the following lines assign those new -permissions to the "User" role, so that normal users can view and edit -"category" objects:: - - p = db.security.getPermission('View', 'category') - db.security.addPermissionToRole('User', p) - - p = db.security.getPermission('Edit', 'category') - db.security.addPermissionToRole('User', p) - -This is all the work that needs to be done for the database. It will store -categories, and let users view and edit them. Now on to the interface -stuff. - -Changing the web left hand frame -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -We need to give the users the ability to create new categories, and the -place to put the link to this functionality is in the left hand function -bar, under the "Issues" area. The file that defines how this area looks is -``html/page``, which is what we are going to be editing next. - -If you look at this file you can see that it contains a lot of "classblock" -sections which are chunks of HTML that will be included or excluded in the -output depending on whether the condition in the classblock is met. Under -the end of the classblock for issue is where we are going to add the -category code:: - - <p class="classblock" - tal:condition="python:request.user.hasPermission('View', 'category')"> - <b>Categories</b><br> - <a tal:condition="python:request.user.hasPermission('Edit', 'category')" - href="category?:template=item">New Category<br></a> - </p> - -The first two lines is the classblock definition, which sets up a condition -that only users who have "View" permission to the "category" object will -have this section included in their output. Next comes a plain "Categories" -header in bold. Everyone who can view categories will get that. - -Next comes the link to the editing area of categories. This link will only -appear if the condition is matched: that condition being that the user has -"Edit" permissions for the "category" objects. If they do have permission -then they will get a link to another page which will let the user add new -categories. - -Note that if you have permission to view but not edit categories then all -you will see is a "Categories" header with nothing underneath it. This is -obviously not very good interface design, but will do for now. I just claim -that it is so I can add more links in this section later on. However to fix -the problem you could change the condition in the classblock statement, so -that only users with "Edit" permission would see the "Categories" stuff. - -Setting up a page to edit categories -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -We defined code in the previous section which let users with the -appropriate permissions see a link to a page which would let them edit -conditions. Now we have to write that page. - -The link was for the item template for the category object. This translates -into the system looking for a file called ``category.item`` in the ``html`` -tracker directory. This is the file that we are going to write now. - -First we add an info tag in a comment which doesn't affect the outcome -of the code at all but is useful for debugging. If you load a page in a -browser and look at the page source, you can see which sections come -from which files by looking for these comments:: - - <!-- category.item --> - -Next we need to add in the METAL macro stuff so we get the normal page -trappings:: - - <tal:block metal:use-macro="templates/page/macros/icing"> - <title metal:fill-slot="head_title">Category editing</title> - <td class="page-header-top" metal:fill-slot="body_title"> - <h2>Category editing</h2> - </td> - <td class="content" metal:fill-slot="content"> - -Next we need to setup up a standard HTML form, which is the whole -purpose of this file. We link to some handy javascript which sends the form -through only once. This is to stop users hitting the send button -multiple times when they are impatient and thus having the form sent -multiple times:: - - <form method="POST" onSubmit="return submit_once()" - enctype="multipart/form-data"> - -Next we define some code which sets up the minimum list of fields that we -require the user to enter. There will be only one field, that of "name", so -they user better put something in it otherwise the whole form is pointless:: - - <input type="hidden" name=":required" value="name"> - -To get everything to line up properly we will put everything in a table, -and put a nice big header on it so the user has an idea what is happening:: - - <table class="form"> - <tr><th class="header" colspan=2>Category</th></tr> - -Next we need the actual field that the user is going to enter the new -category. The "context.name.field(size=60)" bit tells roundup to generate a -normal HTML field of size 60, and the contents of that field will be the -"name" variable of the current context (which is "category"). The upshot of -this is that when the user types something in to the form, a new category -will be created with that name:: - - <tr> - <th nowrap>Name</th> - <td tal:content="structure python:context.name.field(size=60)">name</td> - </tr> - -Then a submit button so that the user can submit the new category:: - - <tr> - <td> </td> - <td colspan=3 tal:content="structure context/submit"> - submit button will go here - </td> - </tr> - -Finally we finish off the tags we used at the start to do the METAL stuff:: - - </td> - </tal:block> - -So putting it all together, and closing the table and form we get:: - - <!-- category.item --> - <tal:block metal:use-macro="templates/page/macros/icing"> - <title metal:fill-slot="head_title">Category editing</title> - <td class="page-header-top" metal:fill-slot="body_title"> - <h2>Category editing</h2> - </td> - <td class="content" metal:fill-slot="content"> - <form method="POST" onSubmit="return submit_once()" - enctype="multipart/form-data"> - - <input type="hidden" name=":required" value="name"> - - <table class="form"> - <tr><th class="header" colspan=2>Category</th></tr> - - <tr> - <th nowrap>Name</th> - <td tal:content="structure python:context.name.field(size=60)">name</td> - </tr> - - <tr> - <td> </td> - <td colspan=3 tal:content="structure context/submit"> - submit button will go here - </td> - </tr> - </table> - </form> - </td> - </tal:block> - -This is quite a lot to just ask the user one simple question, but -there is a lot of setup for basically one line (the form line) to do -its work. To add another field to "category" would involve one more line -(well maybe a few extra to get the formatting correct). - -Adding the category to the issue -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -We now have the ability to create issues to our hearts content, but -that is pointless unless we can assign categories to issues. Just like -the ``html/category.item`` file was used to define how to add a new -category, the ``html/issue.item`` is used to define how a new issue is -created. - -Just like ``category.issue`` this file defines a form which has a table to lay -things out. It doesn't matter where in the table we add new stuff, -it is entirely up to your sense of aesthetics:: - - <th nowrap>Category</th> - <td><span tal:replace="structure context/category/field" /> - <span tal:replace="structure db/category/classhelp" /> - </td> - -First we define a nice header so that the user knows what the next section -is, then the middle line does what we are most interested in. This -``context/category/field`` gets replaced with a field which contains the -category in the current context (the current context being the new issue). - -The classhelp lines generate a link (labelled "list") to a popup window -which contains the list of currently known categories. - -Searching on categories -~~~~~~~~~~~~~~~~~~~~~~~ - -We can add categories, and create issues with categories. The next obvious -thing that we would like to be would be to search issues based on their -category, so that any one working on the web server could look at all -issues in the category "Web" for example. - -If you look in the html/page file and look for the "Search Issues" you will -see that it looks something like ``<a href="issue?:template=search">Search -Issues</a>`` which shows us that when you click on "Search Issues" it will -be looking for a ``issue.search`` file to display. So that is indeed the file -that we are going to change. - -If you look at this file it should be starting to seem familiar. It is a -simple HTML form using a table to define structure. You can add the new -category search code anywhere you like within that form:: - - <tr> - <th>Category:</th> - <td> - <select name="category"> - <option value="">don't care</option> - <option value="">------------</option> - <option tal:repeat="s db/category/list" tal:attributes="value s/name" - tal:content="s/name">category to filter on</option> - </select> - </td> - <td><input type="checkbox" name=":columns" value="category" checked></td> - <td><input type="radio" name=":sort" value="category"></td> - <td><input type="radio" name=":group" value="category"></td> - </tr> - -Most of this is straightforward to anyone who knows HTML. It is just -setting up a select list followed by a checkbox and a couple of radio -buttons. - -The ``tal:repeat`` part repeats the tag for every item in the "category" -table and setting "s" to be each category in turn. - -The ``tal:attributes`` part is setting up the ``value=`` part of the option tag -to be the name part of "s" which is the current category in the loop. - -The ``tal:content`` part is setting the contents of the option tag to be the -name part of "s" again. For objects more complex than category, obviously -you would put an id in the value, and the descriptive part in the content; -but for category they are the same. - -Adding category to the default view -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -We can now add categories, add issues with categories, and search issues -based on categories. This is everything that we need to do, however there -is some more icing that we would like. I think the category of an issue is -important enough that it should be displayed by default when listing all -the issues. - -Unfortunately, this is a bit less obvious than the previous steps. The code -defining how the issues look is in ``html/issue.index``. This is a large table -with a form down the bottom for redisplaying and so forth. - -Firstly we need to add an appropriate header to the start of the table:: - - <th tal:condition="request/show/category">Category</th> - -The condition part of this statement is so that if the user has selected -not to see the Category column then they won't. - -The rest of the table is a loop which will go through every issue that -matches the display criteria. The loop variable is "i" - which means that -every issue gets assigned to "i" in turn. - -The new part of code to display the category will look like this:: - - <td tal:condition="request/show/category" tal:content="i/category"></td> - -The condition is the same as above: only display the condition when the -user hasn't asked for it to be hidden. The next part is to set the content -of the cell to be the category part of "i" - the current issue. - -Finally we have to edit ``html/page`` again. This time to tell it that when the -user clicks on "Unnasigned Issues" or "All Issues" that the category should -be displayed. If you scroll down the page file, you can see the links with -lots of options. The option that we are interested in is the ``:columns=`` one -which tells roundup which fields of the issue to display. Simply add -"category" to that list and it all should work. - - -Adding in state transition control ----------------------------------- - -Sometimes tracker admins want to control the states that users may move issues -to. - -1. add a Multilink property to the status class:: - - stat = Class(db, "status", ... , transitions=Multilink('status'), ...) - - and then edit the statuses already created through the web using the - generic class list / CSV editor. - -2. add an auditor module ``checktransition.py`` in your tracker's - ``detectors`` directory:: - - def checktransition(db, cl, nodeid, newvalues): - ''' Check that the desired transition is valid for the "status" - property. - ''' - if not newvalues.has_key('status'): - return - current = cl.get(nodeid, 'status') - new = newvalues['status'] - if new == current: - return - ok = db.status.get(current, 'transitions') - if new not in ok: - raise ValueError, 'Status not allowed to move from "%s" to "%s"'%( - db.status.get(current, 'name'), db.status.get(new, 'name')) - - def init(db): - db.issue.audit('set', checktransition) - -3. in the ``issue.item`` template, change the status editing bit from:: - - <th nowrap>Status</th> - <td tal:content="structure context/status/menu">status</td> - - to:: - - <th nowrap>Status</th> - <td> - <select tal:condition="context/id" name="status"> - <tal:block tal:define="ok context/status/transitions" - tal:repeat="state db/status/list"> - <option tal:condition="python:state.id in ok" - tal:attributes="value state/id; - selected python:state.id == context.status.id" - tal:content="state/name"></option> - </tal:block> - </select> - <tal:block tal:condition="not:context/id" - tal:replace="structure context/status/menu" /> - </td> - - which displays only the allowed status to transition to. - - -Displaying entire message contents in the issue display -------------------------------------------------------- - -Alter the issue.item template section for messages to:: - - <table class="messages" tal:condition="context/messages"> - <tr><th colspan=3 class="header">Messages</th></tr> - <tal:block tal:repeat="msg context/messages/reverse"> - <tr> - <th><a tal:attributes="href string:msg${msg/id}" - tal:content="string:msg${msg/id}"></a></th> - <th tal:content="string:Author: ${msg/author}">author</th> - <th tal:content="string:Date: ${msg/date}">date</th> - </tr> - <tr> - <td colspan="3" class="content"> - <pre tal:content="msg/content">content</pre> - </td> - </tr> - </tal:block> - </table> - -Restricting the list of users that are assignable to a task ------------------------------------------------------------ - -1. In your tracker's "dbinit.py", create a new Role, say "Developer":: - - db.security.addRole(name='Developer', description='A developer') - -2. Just after that, create a new Permission, say "Fixer", specific to "issue":: - - p = db.security.addPermission(name='Fixer', klass='issue', - description='User is allowed to be assigned to fix issues') - -3. Then assign the new Permission to your "Developer" Role:: - - db.security.addPermissionToRole('Developer', p) - -4. In the issue item edit page ("html/issue.item" in your tracker dir), use - the new Permission in restricting the "assignedto" list:: - - <select name="assignedto"> - <option value="-1">- no selection -</option> - <tal:block tal:repeat="user db/user/list"> - <option tal:condition="python:user.hasPermission('Fixer', context.classname)" - tal:attributes="value user/id; - selected python:user.id == context.assignedto" - tal:content="user/realname"></option> - </tal:block> - </select> - -For extra security, you may wish to set up an auditor to enforce the -Permission requirement (install this as "assignedtoFixer.py" in your tracker -"detectors" directory):: - - def assignedtoMustBeFixer(db, cl, nodeid, newvalues): - ''' Ensure the assignedto value in newvalues is a used with the Fixer - Permission - ''' - if not newvalues.has_key('assignedto'): - # don't care - return - - # get the userid - userid = newvalues['assignedto'] - if not db.security.hasPermission('Fixer', userid, cl.classname): - raise ValueError, 'You do not have permission to edit %s'%cl.classname - - def init(db): - db.issue.audit('set', assignedtoMustBeFixer) - db.issue.audit('create', assignedtoMustBeFixer) - -So now, if the edit attempts to set the assignedto to a user that doesn't have -the "Fixer" Permission, the error will be raised. - - -Setting up a "wizard" (or "druid") for controlled adding of issues ------------------------------------------------------------------- - -1. Set up the page templates you wish to use for data input. My wizard - is going to be a two-step process, first figuring out what category of - issue the user is submitting, and then getting details specific to that - category. The first page includes a table of help, explaining what the - category names mean, and then the core of the form:: - - <form method="POST" onSubmit="return submit_once()" - enctype="multipart/form-data"> - <input type="hidden" name=":template" value="add_page1"> - <input type="hidden" name=":action" value="page1submit"> - - <strong>Category:</strong> - <tal:block tal:replace="structure context/category/menu" /> - <input type="submit" value="Continue"> - </form> - - The next page has the usual issue entry information, with the addition of - the following form fragments:: - - <form method="POST" onSubmit="return submit_once()" - enctype="multipart/form-data" tal:condition="context/is_edit_ok" - tal:define="cat request/form/category/value"> - - <input type="hidden" name=":template" value="add_page2"> - <input type="hidden" name=":required" value="title"> - <input type="hidden" name="category" tal:attributes="value cat"> - - . - . - . - </form> - - Note that later in the form, I test the value of "cat" include form - elements that are appropriate. For example:: - - <tal:block tal:condition="python:cat in '6 10 13 14 15 16 17'.split()"> - <tr> - <th nowrap>Operating System</th> - <td tal:content="structure context/os/field"></td> - </tr> - <tr> - <th nowrap>Web Browser</th> - <td tal:content="structure context/browser/field"></td> - </tr> - </tal:block> - - ... the above section will only be displayed if the category is one of 6, - 10, 13, 14, 15, 16 or 17. - -3. Determine what actions need to be taken between the pages - these are - usually to validate user choices and determine what page is next. Now - encode those actions in methods on the interfaces Client class and insert - hooks to those actions in the "actions" attribute on that class, like so:: - - actions = client.Class.actions + ( - ('page1_submit', page1SubmitAction), - ) - - def page1SubmitAction(self): - ''' Verify that the user has selected a category, and then move on - to page 2. - ''' - category = self.form['category'].value - if category == '-1': - self.error_message.append('You must select a category of report') - return - # everything's ok, move on to the next page - self.template = 'add_page2' - -4. Use the usual "new" action as the :action on the final page, and you're - done (the standard context/submit method can do this for you). - -------------------- - -Back to `Table of Contents`_ - -.. _`Table of Contents`: index.html -
--- a/doc/design.txt Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1642 +0,0 @@ -======================================================== -Roundup - An Issue-Tracking System for Knowledge Workers -======================================================== - -:Authors: Ka-Ping Yee (original__), Richard Jones (implementation) - -__ spec.html - -.. contents:: - -Introduction ---------------- - -This document presents a description of the components -of the Roundup system and specifies their interfaces and -behaviour in sufficient detail to guide an implementation. -For the philosophy and rationale behind the Roundup design, -see the first-round Software Carpentry submission for Roundup. -This document fleshes out that design as well as specifying -interfaces so that the components can be developed separately. - - -The Layer Cake ------------------ - -Lots of software design documents come with a picture of -a cake. Everybody seems to like them. I also like cakes -(i think they are tasty). So i, too, shall include -a picture of a cake here:: - - _________________________________________________________________________ - | E-mail Client | Web Browser | Detector Scripts | Shell | - |------------------+-----------------+----------------------+-------------| - | E-mail User | Web User | Detector | Command | - |-------------------------------------------------------------------------| - | Roundup Database Layer | - |-------------------------------------------------------------------------| - | Hyperdatabase Layer | - |-------------------------------------------------------------------------| - | Storage Layer | - ------------------------------------------------------------------------- - -The colourful parts of the cake are part of our system; -the faint grey parts of the cake are external components. - -I will now proceed to forgo all table manners and -eat from the bottom of the cake to the top. You may want -to stand back a bit so you don't get covered in crumbs. - - -Hyperdatabase -------------- - -The lowest-level component to be implemented is the hyperdatabase. -The hyperdatabase is intended to be -a flexible data store that can hold configurable data in -records which we call items. - -The hyperdatabase is implemented on top of the storage layer, -an external module for storing its data. The storage layer could -be a third-party RDBMS; for a "batteries-included" distribution, -implementing the hyperdatabase on the standard bsddb -module is suggested. - -Dates and Date Arithmetic -~~~~~~~~~~~~~~~~~~~~~~~~~ - -Before we get into the hyperdatabase itself, we need a -way of handling dates. The hyperdatabase module provides -Timestamp objects for -representing date-and-time stamps and Interval objects for -representing date-and-time intervals. - -As strings, date-and-time stamps are specified with -the date in international standard format -(``yyyy-mm-dd``) -joined to the time (``hh:mm:ss``) -by a period "``.``". Dates in -this form can be easily compared and are fairly readable -when printed. An example of a valid stamp is -"``2000-06-24.13:03:59``". -We'll call this the "full date format". When Timestamp objects are -printed as strings, they appear in the full date format with -the time always given in GMT. The full date format is always -exactly 19 characters long. - -For user input, some partial forms are also permitted: -the whole time or just the seconds may be omitted; and the whole date -may be omitted or just the year may be omitted. If the time is given, -the time is interpreted in the user's local time zone. -The Date constructor takes care of these conversions. -In the following examples, suppose that ``yyyy`` is the current year, -``mm`` is the current month, and ``dd`` is the current -day of the month; and suppose that the user is on Eastern Standard Time. - -- "2000-04-17" means <Date 2000-04-17.00:00:00> -- "01-25" means <Date yyyy-01-25.00:00:00> -- "2000-04-17.03:45" means <Date 2000-04-17.08:45:00> -- "08-13.22:13" means <Date yyyy-08-14.03:13:00> -- "11-07.09:32:43" means <Date yyyy-11-07.14:32:43> -- "14:25" means -- <Date yyyy-mm-dd.19:25:00> -- "8:47:11" means -- <Date yyyy-mm-dd.13:47:11> -- the special date "." means "right now" - - -Date intervals are specified using the suffixes -"y", "m", and "d". The suffix "w" (for "week") means 7 days. -Time intervals are specified in hh:mm:ss format (the seconds -may be omitted, but the hours and minutes may not). - -- "3y" means three years -- "2y 1m" means two years and one month -- "1m 25d" means one month and 25 days -- "2w 3d" means two weeks and three days -- "1d 2:50" means one day, two hours, and 50 minutes -- "14:00" means 14 hours -- "0:04:33" means four minutes and 33 seconds - - -The Date class should understand simple date expressions of the form -*stamp* ``+`` *interval* and *stamp* ``-`` *interval*. -When adding or subtracting intervals involving months or years, the -components are handled separately. For example, when evaluating -"``2000-06-25 + 1m 10d``", we first add one month to -get 2000-07-25, then add 10 days to get -2000-08-04 (rather than trying to decide whether -1m 10d means 38 or 40 or 41 days). - -Here is an outline of the Date and Interval classes:: - - class Date: - def __init__(self, spec, offset): - """Construct a date given a specification and a time zone offset. - - 'spec' is a full date or a partial form, with an optional - added or subtracted interval. 'offset' is the local time - zone offset from GMT in hours. - """ - - def __add__(self, interval): - """Add an interval to this date to produce another date.""" - - def __sub__(self, interval): - """Subtract an interval from this date to produce another date.""" - - def __cmp__(self, other): - """Compare this date to another date.""" - - def __str__(self): - """Return this date as a string in the yyyy-mm-dd.hh:mm:ss format.""" - - def local(self, offset): - """Return this date as yyyy-mm-dd.hh:mm:ss in a local time zone.""" - - class Interval: - def __init__(self, spec): - """Construct an interval given a specification.""" - - def __cmp__(self, other): - """Compare this interval to another interval.""" - - def __str__(self): - """Return this interval as a string.""" - - - -Here are some examples of how these classes would behave in practice. -For the following examples, assume that we are on Eastern Standard -Time and the current local time is 19:34:02 on 25 June 2000:: - - >>> Date(".") - <Date 2000-06-26.00:34:02> - >>> _.local(-5) - "2000-06-25.19:34:02" - >>> Date(". + 2d") - <Date 2000-06-28.00:34:02> - >>> Date("1997-04-17", -5) - <Date 1997-04-17.00:00:00> - >>> Date("01-25", -5) - <Date 2000-01-25.00:00:00> - >>> Date("08-13.22:13", -5) - <Date 2000-08-14.03:13:00> - >>> Date("14:25", -5) - <Date 2000-06-25.19:25:00> - >>> Interval(" 3w 1 d 2:00") - <Interval 22d 2:00> - >>> Date(". + 2d") - Interval("3w") - <Date 2000-06-07.00:34:02> - -Items and Classes -~~~~~~~~~~~~~~~~~ - -Items contain data in properties. To Python, these -properties are presented as the key-value pairs of a dictionary. -Each item belongs to a class which defines the names -and types of its properties. The database permits the creation -and modification of classes as well as items. - -Identifiers and Designators -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Each item has a numeric identifier which is unique among -items in its class. The items are numbered sequentially -within each class in order of creation, starting from 1. -The designator -for an item is a way to identify an item in the database, and -consists of the name of the item's class concatenated with -the item's numeric identifier. - -For example, if "spam" and "eggs" are classes, the first -item created in class "spam" has id 1 and designator "spam1". -The first item created in class "eggs" also has id 1 but has -the distinct designator "eggs1". Item designators are -conventionally enclosed in square brackets when mentioned -in plain text. This permits a casual mention of, say, -"[patch37]" in an e-mail message to be turned into an active -hyperlink. - -Property Names and Types -~~~~~~~~~~~~~~~~~~~~~~~~ - -Property names must begin with a letter. - -A property may be one of five basic types: - -- String properties are for storing arbitrary-length strings. - -- Boolean properties are for storing true/false, or yes/no values. - -- Number properties are for storing numeric values. - -- Date properties store date-and-time stamps. - Their values are Timestamp objects. - -- A Link property refers to a single other item - selected from a specified class. The class is part of the property; - the value is an integer, the id of the chosen item. - -- A Multilink property refers to possibly many items - in a specified class. The value is a list of integers. - -*None* is also a permitted value for any of these property -types. An attempt to store None into a Multilink property stores an empty list. - -A property that is not specified will return as None from a *get* -operation. - -Hyperdb Interface Specification -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -TODO: replace the Interface Specifications with links to the pydoc - -The hyperdb module provides property objects to designate -the different kinds of properties. These objects are used when -specifying what properties belong in classes:: - - class String: - def __init__(self, indexme='no'): - """An object designating a String property.""" - - class Boolean: - def __init__(self): - """An object designating a Boolean property.""" - - class Number: - def __init__(self): - """An object designating a Number property.""" - - class Date: - def __init__(self): - """An object designating a Date property.""" - - class Link: - def __init__(self, classname, do_journal='yes'): - """An object designating a Link property that links to - items in a specified class. - - If the do_journal argument is not 'yes' then changes to - the property are not journalled in the linked item. - """ - - class Multilink: - def __init__(self, classname, do_journal='yes'): - """An object designating a Multilink property that links - to items in a specified class. - - If the do_journal argument is not 'yes' then changes to - the property are not journalled in the linked item(s). - """ - - -Here is the interface provided by the hyperdatabase:: - - class Database: - """A database for storing records containing flexible data types.""" - - def __init__(self, config, journaltag=None): - """Open a hyperdatabase given a specifier to some storage. - - The 'storagelocator' is obtained from config.DATABASE. - The meaning of 'storagelocator' depends on the particular - implementation of the hyperdatabase. It could be a file name, - a directory path, a socket descriptor for a connection to a - database over the network, etc. - - The 'journaltag' is a token that will be attached to the journal - entries for any edits done on the database. If 'journaltag' is - None, the database is opened in read-only mode: the Class.create(), - Class.set(), and Class.retire() methods are disabled. - """ - - def __getattr__(self, classname): - """A convenient way of calling self.getclass(classname).""" - - def getclasses(self): - """Return a list of the names of all existing classes.""" - - def getclass(self, classname): - """Get the Class object representing a particular class. - - If 'classname' is not a valid class name, a KeyError is raised. - """ - - class Class: - """The handle to a particular class of items in a hyperdatabase.""" - - def __init__(self, db, classname, **properties): - """Create a new class with a given name and property specification. - - 'classname' must not collide with the name of an existing class, - or a ValueError is raised. The keyword arguments in 'properties' - must map names to property objects, or a TypeError is raised. - """ - - # Editing items: - - def create(self, **propvalues): - """Create a new item of this class and return its id. - - The keyword arguments in 'propvalues' map property names to values. - The values of arguments must be acceptable for the types of their - corresponding properties or a TypeError is raised. If this class - has a key property, it must be present and its value must not - collide with other key strings or a ValueError is raised. Any other - properties on this class that are missing from the 'propvalues' - dictionary are set to None. If an id in a link or multilink - property does not refer to a valid item, an IndexError is raised. - """ - - def get(self, itemid, propname): - """Get the value of a property on an existing item of this class. - - 'itemid' must be the id of an existing item of this class or an - IndexError is raised. 'propname' must be the name of a property - of this class or a KeyError is raised. - """ - - def set(self, itemid, **propvalues): - """Modify a property on an existing item of this class. - - 'itemid' must be the id of an existing item of this class or an - IndexError is raised. Each key in 'propvalues' must be the name - of a property of this class or a KeyError is raised. All values - in 'propvalues' must be acceptable types for their corresponding - properties or a TypeError is raised. If the value of the key - property is set, it must not collide with other key strings or a - ValueError is raised. If the value of a Link or Multilink - property contains an invalid item id, a ValueError is raised. - """ - - def retire(self, itemid): - """Retire an item. - - The properties on the item remain available from the get() method, - and the item's id is never reused. Retired items are not returned - by the find(), list(), or lookup() methods, and other items may - reuse the values of their key properties. - """ - - def history(self, itemid): - """Retrieve the journal of edits on a particular item. - - 'itemid' must be the id of an existing item of this class or an - IndexError is raised. - - The returned list contains tuples of the form - - (date, tag, action, params) - - 'date' is a Timestamp object specifying the time of the change and - 'tag' is the journaltag specified when the database was opened. - 'action' may be: - - 'create' or 'set' -- 'params' is a dictionary of property values - 'link' or 'unlink' -- 'params' is (classname, itemid, propname) - 'retire' -- 'params' is None - """ - - # Locating items: - - def setkey(self, propname): - """Select a String property of this class to be the key property. - - 'propname' must be the name of a String property of this class or - None, or a TypeError is raised. The values of the key property on - all existing items must be unique or a ValueError is raised. - """ - - def getkey(self): - """Return the name of the key property for this class or None.""" - - def lookup(self, keyvalue): - """Locate a particular item by its key property and return its id. - - If this class has no key property, a TypeError is raised. If the - 'keyvalue' matches one of the values for the key property among - the items in this class, the matching item's id is returned; - otherwise a KeyError is raised. - """ - - def find(self, propname, itemid): - """Get the ids of items in this class which link to the given items. - - 'propspec' consists of keyword args propname={itemid:1,} - 'propname' must be the name of a property in this class, or a - KeyError is raised. That property must be a Link or Multilink - property, or a TypeError is raised. - - Any item in this class whose 'propname' property links to any of the - itemids will be returned. Used by the full text indexing, which - knows that "foo" occurs in msg1, msg3 and file7, so we have hits - on these issues: - - db.issue.find(messages={'1':1,'3':1}, files={'7':1}) - """ - - def filter(self, search_matches, filterspec, sort, group): - ''' Return a list of the ids of the active items in this class that - match the 'filter' spec, sorted by the group spec and then the - sort spec. - ''' - - def list(self): - """Return a list of the ids of the active items in this class.""" - - def count(self): - """Get the number of items in this class. - - If the returned integer is 'numitems', the ids of all the items - in this class run from 1 to numitems, and numitems+1 will be the - id of the next item to be created in this class. - """ - - # Manipulating properties: - - def getprops(self): - """Return a dictionary mapping property names to property objects.""" - - def addprop(self, **properties): - """Add properties to this class. - - The keyword arguments in 'properties' must map names to property - objects, or a TypeError is raised. None of the keys in 'properties' - may collide with the names of existing properties, or a ValueError - is raised before any properties have been added. - """ - - def getitem(self, itemid, cache=1): - ''' Return a Item convenience wrapper for the item. - - 'itemid' must be the id of an existing item of this class or an - IndexError is raised. - - 'cache' indicates whether the transaction cache should be queried - for the item. If the item has been modified and you need to - determine what its values prior to modification are, you need to - set cache=0. - ''' - - class Item: - ''' A convenience wrapper for the given item. It provides a mapping - interface to a single item's properties - ''' - -Hyperdatabase Implementations -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Hyperdatabase implementations exist to create the interface described in the -`hyperdb interface specification`_ -over an existing storage mechanism. Examples are relational databases, -\*dbm key-value databases, and so on. - -Several implementations are provided - they belong in the roundup.backends -package. - - -Application Example -~~~~~~~~~~~~~~~~~~~ - -Here is an example of how the hyperdatabase module would work in practice:: - - >>> import hyperdb - >>> db = hyperdb.Database("foo.db", "ping") - >>> db - <hyperdb.Database "foo.db" opened by "ping"> - >>> hyperdb.Class(db, "status", name=hyperdb.String()) - <hyperdb.Class "status"> - >>> _.setkey("name") - >>> db.status.create(name="unread") - 1 - >>> db.status.create(name="in-progress") - 2 - >>> db.status.create(name="testing") - 3 - >>> db.status.create(name="resolved") - 4 - >>> db.status.count() - 4 - >>> db.status.list() - [1, 2, 3, 4] - >>> db.status.lookup("in-progress") - 2 - >>> db.status.retire(3) - >>> db.status.list() - [1, 2, 4] - >>> hyperdb.Class(db, "issue", title=hyperdb.String(), status=hyperdb.Link("status")) - <hyperdb.Class "issue"> - >>> db.issue.create(title="spam", status=1) - 1 - >>> db.issue.create(title="eggs", status=2) - 2 - >>> db.issue.create(title="ham", status=4) - 3 - >>> db.issue.create(title="arguments", status=2) - 4 - >>> db.issue.create(title="abuse", status=1) - 5 - >>> hyperdb.Class(db, "user", username=hyperdb.Key(), password=hyperdb.String()) - <hyperdb.Class "user"> - >>> db.issue.addprop(fixer=hyperdb.Link("user")) - >>> db.issue.getprops() - {"title": <hyperdb.String>, "status": <hyperdb.Link to "status">, - "user": <hyperdb.Link to "user">} - >>> db.issue.set(5, status=2) - >>> db.issue.get(5, "status") - 2 - >>> db.status.get(2, "name") - "in-progress" - >>> db.issue.get(5, "title") - "abuse" - >>> db.issue.find("status", db.status.lookup("in-progress")) - [2, 4, 5] - >>> db.issue.history(5) - [(<Date 2000-06-28.19:09:43>, "ping", "create", {"title": "abuse", "status": 1}), - (<Date 2000-06-28.19:11:04>, "ping", "set", {"status": 2})] - >>> db.status.history(1) - [(<Date 2000-06-28.19:09:43>, "ping", "link", ("issue", 5, "status")), - (<Date 2000-06-28.19:11:04>, "ping", "unlink", ("issue", 5, "status"))] - >>> db.status.history(2) - [(<Date 2000-06-28.19:11:04>, "ping", "link", ("issue", 5, "status"))] - - -For the purposes of journalling, when a Multilink property is -set to a new list of items, the hyperdatabase compares the old -list to the new list. -The journal records "unlink" events for all the items that appear -in the old list but not the new list, -and "link" events for -all the items that appear in the new list but not in the old list. - - -Roundup Database ----------------- - -The Roundup database layer is implemented on top of the -hyperdatabase and mediates calls to the database. -Some of the classes in the Roundup database are considered -issue classes. -The Roundup database layer adds detectors and user items, -and on issues it provides mail spools, nosy lists, and superseders. - -Reserved Classes -~~~~~~~~~~~~~~~~ - -Internal to this layer we reserve three special classes -of items that are not issues. - -Users -""""" - -Users are stored in the hyperdatabase as items of -class "user". The "user" class has the definition:: - - hyperdb.Class(db, "user", username=hyperdb.String(), - password=hyperdb.String(), - address=hyperdb.String()) - db.user.setkey("username") - -Messages -"""""""" - -E-mail messages are represented by hyperdatabase items of class "msg". -The actual text content of the messages is stored in separate files. -(There's no advantage to be gained by stuffing them into the -hyperdatabase, and if messages are stored in ordinary text files, -they can be grepped from the command line.) The text of a message is -saved in a file named after the message item designator (e.g. "msg23") -for the sake of the command interface (see below). Attachments are -stored separately and associated with "file" items. -The "msg" class has the definition:: - - hyperdb.Class(db, "msg", author=hyperdb.Link("user"), - recipients=hyperdb.Multilink("user"), - date=hyperdb.Date(), - summary=hyperdb.String(), - files=hyperdb.Multilink("file")) - -The "author" property indicates the author of the message -(a "user" item must exist in the hyperdatabase for any messages -that are stored in the system). -The "summary" property contains a summary of the message for display -in a message index. - -Files -""""" - -Submitted files are represented by hyperdatabase -items of class "file". Like e-mail messages, the file content -is stored in files outside the database, -named after the file item designator (e.g. "file17"). -The "file" class has the definition:: - - hyperdb.Class(db, "file", user=hyperdb.Link("user"), - name=hyperdb.String(), - type=hyperdb.String()) - -The "user" property indicates the user who submitted the -file, the "name" property holds the original name of the file, -and the "type" property holds the MIME type of the file as received. - -Issue Classes -~~~~~~~~~~~~~ - -All issues have the following standard properties: - -=========== ========================== -Property Definition -=========== ========================== -title hyperdb.String() -messages hyperdb.Multilink("msg") -files hyperdb.Multilink("file") -nosy hyperdb.Multilink("user") -superseder hyperdb.Multilink("issue") -=========== ========================== - -Also, two Date properties named "creation" and "activity" are -fabricated by the Roundup database layer. By "fabricated" we -mean that no such properties are actually stored in the -hyperdatabase, but when properties on issues are requested, the -"creation" and "activity" properties are made available. -The value of the "creation" property is the date when an issue was -created, and the value of the "activity" property is the -date when any property on the issue was last edited (equivalently, -these are the dates on the first and last records in the issue's journal). - -Roundupdb Interface Specification -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The interface to a Roundup database delegates most method -calls to the hyperdatabase, except for the following -changes and additional methods:: - - class Database: - def getuid(self): - """Return the id of the "user" item associated with the user - that owns this connection to the hyperdatabase.""" - - class Class: - # Overridden methods: - - def create(self, **propvalues): - def set(self, **propvalues): - def retire(self, itemid): - """These operations trigger detectors and can be vetoed. Attempts - to modify the "creation" or "activity" properties cause a KeyError. - """ - - # New methods: - - def audit(self, event, detector): - def react(self, event, detector): - """Register a detector (see below for more details).""" - - class IssueClass(Class): - # Overridden methods: - - def __init__(self, db, classname, **properties): - """The newly-created class automatically includes the "messages", - "files", "nosy", and "superseder" properties. If the 'properties' - dictionary attempts to specify any of these properties or a - "creation" or "activity" property, a ValueError is raised.""" - - def get(self, itemid, propname): - def getprops(self): - """In addition to the actual properties on the item, these - methods provide the "creation" and "activity" properties.""" - - # New methods: - - def addmessage(self, itemid, summary, text): - """Add a message to an issue's mail spool. - - A new "msg" item is constructed using the current date, the - user that owns the database connection as the author, and - the specified summary text. The "files" and "recipients" - fields are left empty. The given text is saved as the body - of the message and the item is appended to the "messages" - field of the specified issue. - """ - - def sendmessage(self, itemid, msgid): - """Send a message to the members of an issue's nosy list. - - The message is sent only to users on the nosy list who are not - already on the "recipients" list for the message. These users - are then added to the message's "recipients" list. - """ - - -Default Schema -~~~~~~~~~~~~~~ - -The default schema included with Roundup turns it into a -typical software bug tracker. The database is set up like this:: - - pri = Class(db, "priority", name=hyperdb.String(), order=hyperdb.String()) - pri.setkey("name") - pri.create(name="critical", order="1") - pri.create(name="urgent", order="2") - pri.create(name="bug", order="3") - pri.create(name="feature", order="4") - pri.create(name="wish", order="5") - - stat = Class(db, "status", name=hyperdb.String(), order=hyperdb.String()) - stat.setkey("name") - stat.create(name="unread", order="1") - stat.create(name="deferred", order="2") - stat.create(name="chatting", order="3") - stat.create(name="need-eg", order="4") - stat.create(name="in-progress", order="5") - stat.create(name="testing", order="6") - stat.create(name="done-cbb", order="7") - stat.create(name="resolved", order="8") - - Class(db, "keyword", name=hyperdb.String()) - - Class(db, "issue", fixer=hyperdb.Multilink("user"), - topic=hyperdb.Multilink("keyword"), - priority=hyperdb.Link("priority"), - status=hyperdb.Link("status")) - -(The "order" property hasn't been explained yet. It -gets used by the Web user interface for sorting.) - -The above isn't as pretty-looking as the schema specification -in the first-stage submission, but it could be made just as easy -with the addition of a convenience function like Choice -for setting up the "priority" and "status" classes:: - - def Choice(name, *options): - cl = Class(db, name, name=hyperdb.String(), order=hyperdb.String()) - for i in range(len(options)): - cl.create(name=option[i], order=i) - return hyperdb.Link(name) - - -Detector Interface ------------------- - -Detectors are Python functions that are triggered on certain -kinds of events. The definitions of the -functions live in Python modules placed in a directory set aside -for this purpose. Importing the Roundup database module also -imports all the modules in this directory, and the ``init()`` -function of each module is called when a database is opened to -provide it a chance to register its detectors. - -There are two kinds of detectors: - -1. an auditor is triggered just before modifying an item -2. a reactor is triggered just after an item has been modified - -When the Roundup database is about to perform a -``create()``, ``set()``, or ``retire()`` -operation, it first calls any *auditors* that -have been registered for that operation on that class. -Any auditor may raise a *Reject* exception -to abort the operation. - -If none of the auditors raises an exception, the database -proceeds to carry out the operation. After it's done, it -then calls all of the *reactors* that have been registered -for the operation. - -Detector Interface Specification -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The ``audit()`` and ``react()`` methods -register detectors on a given class of items:: - - class Class: - def audit(self, event, detector): - """Register an auditor on this class. - - 'event' should be one of "create", "set", or "retire". - 'detector' should be a function accepting four arguments. - """ - - def react(self, event, detector): - """Register a reactor on this class. - - 'event' should be one of "create", "set", or "retire". - 'detector' should be a function accepting four arguments. - """ - -Auditors are called with the arguments:: - - audit(db, cl, itemid, newdata) - -where ``db`` is the database, ``cl`` is an -instance of Class or IssueClass within the database, and ``newdata`` -is a dictionary mapping property names to values. - -For a ``create()`` -operation, the ``itemid`` argument is None and newdata -contains all of the initial property values with which the item -is about to be created. - -For a ``set()`` operation, newdata -contains only the names and values of properties that are about -to be changed. - -For a ``retire()`` operation, newdata is None. - -Reactors are called with the arguments:: - - react(db, cl, itemid, olddata) - -where ``db`` is the database, ``cl`` is an -instance of Class or IssueClass within the database, and ``olddata`` -is a dictionary mapping property names to values. - -For a ``create()`` -operation, the ``itemid`` argument is the id of the -newly-created item and ``olddata`` is None. - -For a ``set()`` operation, ``olddata`` -contains the names and previous values of properties that were changed. - -For a ``retire()`` operation, ``itemid`` is the -id of the retired item and ``olddata`` is None. - -Detector Example -~~~~~~~~~~~~~~~~ - -Here is an example of detectors written for a hypothetical -project-management application, where users can signal approval -of a project by adding themselves to an "approvals" list, and -a project proceeds when it has three approvals:: - - # Permit users only to add themselves to the "approvals" list. - - def check_approvals(db, cl, id, newdata): - if newdata.has_key("approvals"): - if cl.get(id, "status") == db.status.lookup("approved"): - raise Reject, "You can't modify the approvals list " \ - "for a project that has already been approved." - old = cl.get(id, "approvals") - new = newdata["approvals"] - for uid in old: - if uid not in new and uid != db.getuid(): - raise Reject, "You can't remove other users from the " - "approvals list; you can only remove yourself." - for uid in new: - if uid not in old and uid != db.getuid(): - raise Reject, "You can't add other users to the approvals " - "list; you can only add yourself." - - # When three people have approved a project, change its - # status from "pending" to "approved". - - def approve_project(db, cl, id, olddata): - if olddata.has_key("approvals") and len(cl.get(id, "approvals")) == 3: - if cl.get(id, "status") == db.status.lookup("pending"): - cl.set(id, status=db.status.lookup("approved")) - - def init(db): - db.project.audit("set", check_approval) - db.project.react("set", approve_project) - -Here is another example of a detector that can allow or prevent -the creation of new items. In this scenario, patches for a software -project are submitted by sending in e-mail with an attached file, -and we want to ensure that there are text/plain attachments on -the message. The maintainer of the package can then apply the -patch by setting its status to "applied":: - - # Only accept attempts to create new patches that come with patch files. - - def check_new_patch(db, cl, id, newdata): - if not newdata["files"]: - raise Reject, "You can't submit a new patch without " \ - "attaching a patch file." - for fileid in newdata["files"]: - if db.file.get(fileid, "type") != "text/plain": - raise Reject, "Submitted patch files must be text/plain." - - # When the status is changed from "approved" to "applied", apply the patch. - - def apply_patch(db, cl, id, olddata): - if cl.get(id, "status") == db.status.lookup("applied") and \ - olddata["status"] == db.status.lookup("approved"): - # ...apply the patch... - - def init(db): - db.patch.audit("create", check_new_patch) - db.patch.react("set", apply_patch) - - -Command Interface ------------------ - -The command interface is a very simple and minimal interface, -intended only for quick searches and checks from the shell prompt. -(Anything more interesting can simply be written in Python using -the Roundup database module.) - -Command Interface Specification -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -A single command, roundup, provides basic access to -the hyperdatabase from the command line:: - - roundup-admin help - roundup-admin get [-list] designator[, designator,...] propname - roundup-admin set designator[, designator,...] propname=value ... - roundup-admin find [-list] classname propname=value ... - -See ``roundup-admin help commands`` for a complete list of commands. - -Property values are represented as strings in command arguments -and in the printed results: - -- Strings are, well, strings. - -- Numbers are displayed the same as strings. - -- Booleans are displayed as 'Yes' or 'No'. - -- Date values are printed in the full date format in the local - time zone, and accepted in the full format or any of the partial - formats explained above. - -- Link values are printed as item designators. When given as - an argument, item designators and key strings are both accepted. - -- Multilink values are printed as lists of item designators - joined by commas. When given as an argument, item designators - and key strings are both accepted; an empty string, a single item, - or a list of items joined by commas is accepted. - -When multiple items are specified to the -roundup get or roundup set -commands, the specified properties are retrieved or set -on all the listed items. - -When multiple results are returned by the roundup get -or roundup find commands, they are printed one per -line (default) or joined by commas (with the -list) option. - -Usage Example -~~~~~~~~~~~~~ - -To find all messages regarding in-progress issues that -contain the word "spam", for example, you could execute the -following command from the directory where the database -dumps its files:: - - shell% for issue in `roundup find issue status=in-progress`; do - > grep -l spam `roundup get $issue messages` - > done - msg23 - msg49 - msg50 - msg61 - shell% - -Or, using the -list option, this can be written as a single command:: - - shell% grep -l spam `roundup get \ - \`roundup find -list issue status=in-progress\` messages` - msg23 - msg49 - msg50 - msg61 - shell% - - -E-mail User Interface ---------------------- - -The Roundup system must be assigned an e-mail address -at which to receive mail. Messages should be piped to -the Roundup mail-handling script by the mail delivery -system (e.g. using an alias beginning with "|" for sendmail). - -Message Processing -~~~~~~~~~~~~~~~~~~ - -Incoming messages are examined for multiple parts. -In a multipart/mixed message or part, each subpart is -extracted and examined. In a multipart/alternative -message or part, we look for a text/plain subpart and -ignore the other parts. The text/plain subparts are -assembled to form the textual body of the message, to -be stored in the file associated with a "msg" class item. -Any parts of other types are each stored in separate -files and given "file" class items that are linked to -the "msg" item. - -The "summary" property on message items is taken from -the first non-quoting section in the message body. -The message body is divided into sections by blank lines. -Sections where the second and all subsequent lines begin -with a ">" or "|" character are considered "quoting -sections". The first line of the first non-quoting -section becomes the summary of the message. - -All of the addresses in the To: and Cc: headers of the -incoming message are looked up among the user items, and -the corresponding users are placed in the "recipients" -property on the new "msg" item. The address in the From: -header similarly determines the "author" property of the -new "msg" item. -The default handling for -addresses that don't have corresponding users is to create -new users with no passwords and a username equal to the -address. (The web interface does not permit logins for -users with no passwords.) If we prefer to reject mail from -outside sources, we can simply register an auditor on the -"user" class that prevents the creation of user items with -no passwords. - -The subject line of the incoming message is examined to -determine whether the message is an attempt to create a new -issue or to discuss an existing issue. A designator enclosed -in square brackets is sought as the first thing on the -subject line (after skipping any "Fwd:" or "Re:" prefixes). - -If an issue designator (class name and id number) is found -there, the newly created "msg" item is added to the "messages" -property for that issue, and any new "file" items are added to -the "files" property for the issue. - -If just an issue class name is found there, we attempt to -create a new issue of that class with its "messages" property -initialized to contain the new "msg" item and its "files" -property initialized to contain any new "file" items. - -Both cases may trigger detectors (in the first case we -are calling the set() method to add the message to the -issue's spool; in the second case we are calling the -create() method to create a new item). If an auditor -raises an exception, the original message is bounced back to -the sender with the explanatory message given in the exception. - -Nosy Lists -~~~~~~~~~~ - -A standard detector is provided that watches for additions -to the "messages" property. When a new message is added, the -detector sends it to all the users on the "nosy" list for the -issue that are not already on the "recipients" list of the -message. Those users are then appended to the "recipients" -property on the message, so multiple copies of a message -are never sent to the same user. The journal recorded by -the hyperdatabase on the "recipients" property then provides -a log of when the message was sent to whom. - -Setting Properties -~~~~~~~~~~~~~~~~~~ - -The e-mail interface also provides a simple way to set -properties on issues. At the end of the subject line, -``propname=value`` pairs can be -specified in square brackets, using the same conventions -as for the roundup ``set`` shell command. - - -Web User Interface ------------------- - -The web interface is provided by a CGI script that can be -run under any web server. A simple web server can easily be -built on the standard CGIHTTPServer module, and -should also be included in the distribution for quick -out-of-the-box deployment. - -The user interface is constructed from a number of template -files containing mostly HTML. Among the HTML tags in templates -are interspersed some nonstandard tags, which we use as -placeholders to be replaced by properties and their values. - -Views and View Specifiers -~~~~~~~~~~~~~~~~~~~~~~~~~ - -There are two main kinds of views: *index* views and *issue* views. -An index view displays a list of issues of a particular class, -optionally sorted and filtered as requested. An issue view -presents the properties of a particular issue for editing -and displays the message spool for the issue. - -A view specifier is a string that specifies -all the options needed to construct a particular view. -It goes after the URL to the Roundup CGI script or the -web server to form the complete URL to a view. When the -result of selecting a link or submitting a form takes -the user to a new view, the Web browser should be redirected -to a canonical location containing a complete view specifier -so that the view can be bookmarked. - -Displaying Properties -~~~~~~~~~~~~~~~~~~~~~ - -Properties appear in the user interface in three contexts: -in indices, in editors, and as filters. For each type of -property, there are several display possibilities. For example, -in an index view, a string property may just be printed as -a plain string, but in an editor view, that property should -be displayed in an editable field. - -The display of a property is handled by functions in -a displayers module. Each function accepts at -least three standard arguments -- the database, class name, -and item id -- and returns a chunk of HTML. - -Displayer functions are triggered by <display> -tags in templates. The call attribute of the tag -provides a Python expression for calling the displayer -function. The three standard arguments are inserted in -front of the arguments given. For example, the occurrence of:: - - <display call="plain('status', max=30)"> - -in a template triggers a call to:: - - plain(db, "issue", 13, "status", max=30) - - -when displaying issue 13 in the "issue" class. The displayer -functions can accept extra arguments to further specify -details about the widgets that should be generated. By defining new -displayer functions, the user interface can be highly customized. - -Some of the standard displayer functions include: - -========= ==================================================================== -Function Description -========= ==================================================================== -plain display a String property directly; - display a Date property in a specified time zone with an option - to omit the time from the date stamp; for a Link or Multilink - property, display the key strings of the linked items (or the - ids if the linked class has no key property) -field display a property like the - plain displayer above, but in a text field - to be edited -menu for a Link property, display - a menu of the available choices -link for a Link or Multilink property, - display the names of the linked items, hyperlinked to the - issue views on those items -count for a Multilink property, display - a count of the number of links in the list -reldate display a Date property in terms - of an interval relative to the current date (e.g. "+ 3w", "- 2d"). -download show a Link("file") or Multilink("file") - property using links that allow you to download files -checklist for a Link or Multilink property, - display checkboxes for the available choices to permit filtering -========= ==================================================================== - -TODO: See the htmltemplate pydoc for a complete list of the functions - - -Index Views -~~~~~~~~~~~ - -An index view contains two sections: a filter section -and an index section. -The filter section provides some widgets for selecting -which issues appear in the index. The index section is -a table of issues. - -Index View Specifiers -""""""""""""""""""""" - -An index view specifier looks like this (whitespace -has been added for clarity):: - - /issue?status=unread,in-progress,resolved& - topic=security,ui& - :group=priority& - :sort=-activity& - :filters=status,topic& - :columns=title,status,fixer - - -The index view is determined by two parts of the -specifier: the layout part and the filter part. -The layout part consists of the query parameters that -begin with colons, and it determines the way that the -properties of selected items are displayed. -The filter part consists of all the other query parameters, -and it determines the criteria by which items -are selected for display. - -The filter part is interactively manipulated with -the form widgets displayed in the filter section. The -layout part is interactively manipulated by clicking -on the column headings in the table. - -The filter part selects the union of the -sets of issues with values matching any specified Link -properties and the intersection of the sets -of issues with values matching any specified Multilink -properties. - -The example specifies an index of "issue" items. -Only issues with a "status" of either -"unread" or "in-progres" or "resolved" are displayed, -and only issues with "topic" values including both -"security" and "ui" are displayed. The issues -are grouped by priority, arranged in ascending order; -and within groups, sorted by activity, arranged in -descending order. The filter section shows filters -for the "status" and "topic" properties, and the -table includes columns for the "title", "status", and -"fixer" properties. - -Associated with each issue class is a default -layout specifier. The layout specifier in the above -example is the default layout to be provided with -the default bug-tracker schema described above in -section 4.4. - -Filter Section -"""""""""""""" - -The template for a filter section provides the -filtering widgets at the top of the index view. -Fragments enclosed in ``<property>...</property>`` -tags are included or omitted depending on whether the -view specifier requests a filter for a particular property. - -Here's a simple example of a filter template:: - - <property name=status> - <display call="checklist('status')"> - </property> - <br> - <property name=priority> - <display call="checklist('priority')"> - </property> - <br> - <property name=fixer> - <display call="menu('fixer')"> - </property> - -Index Section -""""""""""""" - -The template for an index section describes one row of -the index table. -Fragments enclosed in ``<property>...</property>`` -tags are included or omitted depending on whether the -view specifier requests a column for a particular property. -The table cells should contain <display> tags -to display the values of the issue's properties. - -Here's a simple example of an index template:: - - <tr> - <property name=title> - <td><display call="plain('title', max=50)"></td> - </property> - <property name=status> - <td><display call="plain('status')"></td> - </property> - <property name=fixer> - <td><display call="plain('fixer')"></td> - </property> - </tr> - -Sorting -""""""" - -String and Date values are sorted in the natural way. -Link properties are sorted according to the value of the -"order" property on the linked items if it is present; or -otherwise on the key string of the linked items; or -finally on the item ids. Multilink properties are -sorted according to how many links are present. - -Issue Views -~~~~~~~~~~~ - -An issue view contains an editor section and a spool section. -At the top of an issue view, links to superseding and superseded -issues are always displayed. - -Issue View Specifiers -""""""""""""""""""""" - -An issue view specifier is simply the issue's designator:: - - /patch23 - - -Editor Section -"""""""""""""" - -The editor section is generated from a template -containing <display> tags to insert -the appropriate widgets for editing properties. - -Here's an example of a basic editor template:: - - <table> - <tr> - <td colspan=2> - <display call="field('title', size=60)"> - </td> - </tr> - <tr> - <td> - <display call="field('fixer', size=30)"> - </td> - <td> - <display call="menu('status')> - </td> - </tr> - <tr> - <td> - <display call="field('nosy', size=30)"> - </td> - <td> - <display call="menu('priority')> - </td> - </tr> - <tr> - <td colspan=2> - <display call="note()"> - </td> - </tr> - </table> - -As shown in the example, the editor template can also -request the display of a "note" field, which is a -text area for entering a note to go along with a change. - -When a change is submitted, the system automatically -generates a message describing the changed properties. -The message displays all of the property values on the -issue and indicates which ones have changed. -An example of such a message might be this:: - - title: Polly Parrot is dead - priority: critical - status: unread -> in-progress - fixer: (none) - keywords: parrot,plumage,perch,nailed,dead - -If a note is given in the "note" field, the note is -appended to the description. The message is then added -to the issue's message spool (thus triggering the standard -detector to react by sending out this message to the nosy list). - -Spool Section -""""""""""""" - -The spool section lists messages in the issue's "messages" -property. The index of messages displays the "date", "author", -and "summary" properties on the message items, and selecting a -message takes you to its content. - -Access Control --------------- - -At each point that requires an action to be performed, the security mechanisms -are asked if the current user has permission. This permission is defined as a -Permission. - -Individual assignment of Permission to user is unwieldy. The concept of a -Role, which encompasses several Permissions and may be assigned to many Users, -is quite well developed in many projects. Roundup will take this path, and -allow the multiple assignment of Roles to Users, and multiple Permissions to -Roles. These definitions are not persistent - they're defined when the -application initialises. - -There will be two levels of Permission. The Class level permissions define -logical permissions associated with all items of a particular class (or all -classes). The Item level permissions define logical permissions associated -with specific items by way of their user-linked properties. - - -Access Control Interface Specification -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The security module defines:: - - class Permission: - ''' Defines a Permission with the attributes - - name - - description - - klass (optional) - - The klass may be unset, indicating that this permission is not - locked to a particular hyperdb class. There may be multiple - Permissions for the same name for different classes. - ''' - - class Role: - ''' Defines a Role with the attributes - - name - - description - - permissions - ''' - - class Security: - def __init__(self, db): - ''' Initialise the permission and role stores, and add in the - base roles (for admin user). - ''' - - def getPermission(self, permission, classname=None): - ''' Find the Permission matching the name and for the class, if the - classname is specified. - - Raise ValueError if there is no exact match. - ''' - - def hasPermission(self, permission, userid, classname=None): - ''' Look through all the Roles, and hence Permissions, and see if - "permission" is there for the specified classname. - ''' - - def hasItemPermission(self, classname, itemid, **propspec): - ''' Check the named properties of the given item to see if the - userid appears in them. If it does, then the user is granted - this permission check. - - 'propspec' consists of a set of properties and values that - must be present on the given item for access to be granted. - - If a property is a Link, the value must match the property - value. If a property is a Multilink, the value must appear - in the Multilink list. - ''' - - def addPermission(self, **propspec): - ''' Create a new Permission with the properties defined in - 'propspec' - ''' - - def addRole(self, **propspec): - ''' Create a new Role with the properties defined in 'propspec' - ''' - - def addPermissionToRole(self, rolename, permission): - ''' Add the permission to the role's permission list. - - 'rolename' is the name of the role to add permission to. - ''' - -Modules such as ``cgi/client.py`` and ``mailgw.py`` define their own -permissions like so (this example is ``cgi/client.py``):: - - def initialiseSecurity(security): - ''' Create some Permissions and Roles on the security object - - This function is directly invoked by security.Security.__init__() - as a part of the Security object instantiation. - ''' - p = security.addPermission(name="Web Registration", - description="Anonymous users may register through the web") - security.addToRole('Anonymous', p) - -Detectors may also define roles in their init() function:: - - def init(db): - # register an auditor that checks that a user has the "May Resolve" - # Permission before allowing them to set an issue status to "resolved" - db.issue.audit('set', checkresolvedok) - p = db.security.addPermission(name="May Resolve", klass="issue") - security.addToRole('Manager', p) - -The tracker dbinit module then has in ``open()``:: - - # open the database - it must be modified to init the Security class - # from security.py as db.security - db = Database(config, name) - - # add some extra permissions and associate them with roles - ei = db.security.addPermission(name="Edit", klass="issue", - description="User is allowed to edit issues") - db.security.addPermissionToRole('User', ei) - ai = db.security.addPermission(name="View", klass="issue", - description="User is allowed to access issues") - db.security.addPermissionToRole('User', ai) - -In the dbinit ``init()``:: - - # create the two default users - user.create(username="admin", password=Password(adminpw), - address=config.ADMIN_EMAIL, roles='Admin') - user.create(username="anonymous", roles='Anonymous') - -Then in the code that matters, calls to ``hasPermission`` and -``hasItemPermission`` are made to determine if the user has permission -to perform some action:: - - if db.security.hasPermission('issue', 'Edit', userid): - # all ok - - if db.security.hasItemPermission('issue', itemid, assignedto=userid): - # all ok - -Code in the core will make use of these methods, as should code in auditors in -custom templates. The htmltemplate will implement a new tag, ``<require>`` -which has the form:: - - <require permission="name,name,name" assignedto="$userid" status="open"> - HTML to display if the user has the permission. - <else> - HTML to display if the user does not have the permission. - </require> - -where: - -- the permission attribute gives a comma-separated list of permission names. - These are checked in turn using ``hasPermission`` and requires one to - be OK. -- the other attributes are lookups on the item using ``hasItemPermission``. If - the attribute value is "$userid" then the current user's userid is tested. - -Any of these tests must pass or the ``<require>`` check will fail. The section -of html within the side of the ``<else>`` that fails is remove from processing. - -Authentication of Users -~~~~~~~~~~~~~~~~~~~~~~~ - -Users must be authenticated correctly for the above controls to work. This is -not done in the current mail gateway at all. Use of digital signing of -messages could alleviate this problem. - -The exact mechanism of registering the digital signature should be flexible, -with perhaps a level of trust. Users who supply their signature through their -first message into the tracker should be at a lower level of trust to those -who supply their signature to an admin for submission to their user details. - - -Anonymous Users -~~~~~~~~~~~~~~~ - -The "anonymous" user must always exist, and defines the access permissions for -anonymous users. Unknown users accessing Roundup through the web or email -interfaces will be logged in as the "anonymous" user. - - -Use Cases -~~~~~~~~~ - -public - end users can submit bugs, request new features, request support - The Users would be given the default "User" Role which gives "View" and - "Edit" Permission to the "issue" class. -developer - developers can fix bugs, implement new features, provide support - A new Role "Developer" is created with the Permission "Fixer" which is - checked for in custom auditors that see whether the issue is being - resolved with a particular resolution ("fixed", "implemented", - "supported") and allows that resolution only if the permission is - available. -manager - approvers/managers can approve new features and signoff bug fixes - A new Role "Manager" is created with the Permission "Signoff" which is - checked for in custom auditors that see whether the issue status is being - changed similar to the developer example. -admin - administrators can add users and set user's roles - The existing Role "Admin" has the Permissions "Edit" for all classes - (including "user") and "Web Roles" which allow the desired actions. -system - automated request handlers running various report/escalation scripts - A combination of existing and new Roles, Permissions and auditors could - be used here. -privacy - issues that are only visible to some users - A new property is added to the issue which marks the user or group of - users who are allowed to view and edit the issue. An auditor will check - for edit access, and the htmltemplate <require> tag can check for view - access. - - -Deployment Scenarios --------------------- - -The design described above should be general enough -to permit the use of Roundup for bug tracking, managing -projects, managing patches, or holding discussions. By -using items of multiple types, one could deploy a system -that maintains requirement specifications, catalogs bugs, -and manages submitted patches, where patches could be -linked to the bugs and requirements they address. - - -Acknowledgements ----------------- - -My thanks are due to Christy Heyl for -reviewing and contributing suggestions to this paper -and motivating me to get it done, and to -Jesse Vincent, Mark Miller, Christopher Simons, -Jeff Dunmall, Wayne Gramlich, and Dean Tribble for -their assistance with the first-round submission. - -Changes to this document ------------------------- - -- Added Boolean and Number types -- Added section Hyperdatabase Implementations -- "Item" has been renamed to "Issue" to account for the more specific nature - of the Class. - -
--- a/doc/developers.txt Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,74 +0,0 @@ -================== -Developing Roundup -================== - -:Version: $Revision: 1.4 $ - -Note: the intended audience of this document is the developers of the core -Roundup code. If you just wish to alter some behaviour of your Roundup -installation, see `customising roundup`_. - -.. contents:: - -Getting Started ---------------- - -Anyone wishing to help in the development of Roundup must read `Roundup's -Design Document`_ and the `implementation notes`_. - -All development is coordinated through two resources: - -- roundup-dev mailing list at - http://lists.sourceforge.net/mailman/listinfo/roundup-devel -- Sourceforge's issue trackers at - https://sourceforge.net/tracker/?group_id=31577 - -Small Changes -------------- - -Most small changes can be submitted through the Feature tracker, with patches -attached that give context diffs of the affected source. - - -CVS Access ----------- - -To get CVS access, contact richard@users.sourceforge.net. - - -Project Rules -------------- - -Mostly the project follows Guido's Style (though naming tends to be a little -relaxed sometimes). In short: - -- 80 column width code -- 4-space indentations -- All modules must have a CVS Id line near the top, and a CVS Log at the end - -Other project rules: - -- New functionality must be documented, even briefly (so at least we know - where there's missing documentation) and changes to tracker configuration - must be logged in the upgrading document. -- subscribe to roundup-checkins to receive checkin notifications from the - other developers with CVS access -- discuss any changes with the other developers on roundup-dev. If nothing - else, this makes sure there's no rude shocks -- write unit tests for changes you make (where possible), and ensure that - all unit tests run before committing changes -- run pychecker over changed code - -The administrators of the project reserve the right to boot developers who -consistently check in code which is either broken or takes the codebase in -directions that have not been agreed to. - - ------------------ - -Back to `Table of Contents`_ - -.. _`Table of Contents`: index.html -.. _`Customising Roundup`: customizing.html -.. _`Roundup's Design Document`: spec.html -.. _`implementation notes`: implementation.html
--- a/doc/getting_started.txt Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,279 +0,0 @@ -=============== -Getting Started -=============== - -:Version: $Revision: 1.7 $ - -.. contents:: - - -The following instructions assume that you have installed roundup. If you -haven't, you may still proceed - just run the commands as -"``PYTHONPATH=. python roundup/scripts/roundup_admin.py``" for -``roundup-admin`` and -"``PYTHONPATH=. python roundup/scripts/roundup_server.py``" for -``roundup-server``. - -The Tracker ------------ - -We'll be referring to the term tracker a lot from now on. A tracker is a -directory in your filesystem that is where all the information about a live -issue -tracker database is stored. The data that is entered as issues, the users who -access the database and the definition of the database itself all reside there: - -Hyperdatabase - This is the lowest component of Roundup and is where all the issues, - users, file attachments and messages are stored. - -Database schema - This describes the content of the hyperdatabase - what fields are stored - for issues, what user information, etc. Being stored in the tracker, - this allows it to be customised for a particular application. It also - means that changes in the Roundup core code do not affect a running - tracker. - -Web Interface - The web interface templates are defined in the tracker too - and the - actual CGI interface class is defined (mostly using base classes in the - Roundup core code) so it, like the database, may be customised for each - tracker in use. - -Trackers are created using the ``roundup-admin`` tool. - -Command Line Tool ------------------ - -To set up a new tracker, run "``roundup-admin install``". You will be -asked a few questions: - -1. Tracker home directory -2. Schema to use -3. Database back-end to use - -Once you've chosen these, roundup will install the tracker for you. It will -then indicate that you should configure some more information in the -file "``config.py``" in the tracker home. It -should be edited before roundup is initialised, and may have the following -variable declarations: - -MAILHOST - The SMTP mail host that roundup will use to send mail -MAIL_DOMAIN - The domain name used for email addresses -TRACKER_WEB - The web address of the issue tracker's web interface - -The email addresses used by the system by default are: - -TRACKER_EMAIL: ``issue_tracker@MAIL_DOMAIN`` - submissions of issues - -ADMIN_EMAIL: ``roundup-admin@MAIL_DOMAIN`` - roundup's internal use (problems, etc) - -You may also alter the default schema - see the `customisation`_ documentation -for more info on both configuration variables and schema modifications. - -Once you're happy (and note that you can change any of this after the tracker -is initialised too!) you must run "``roundup-admin initialise``". - -You should also think about whether there is going to be controlled access -to the -tracker on the machine the tracker is running on. That is, who can -actually make -changes to the database using the roundup-admin tool. See the section on -Users_and_Access_Control for information on how to secure your tracker from the -start. - -E-Mail Interface ----------------- - -Setup 1: As a mail alias pipe process -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Set up a mail alias called "issue_tracker" as (include the quote marks): -"``|/usr/bin/python /usr/local/bin/roundup-mailgw <tracker_home>``" - -In some installations (e.g. RedHat 6.2 I think) you'll need to set up smrsh so -sendmail will accept the pipe command. In that case, symlink -``/etc/smrsh/roundup-mailgw`` to "``/usr/local/bin/roundup-mailgw``" and change -the command to:: - - |roundup-mailgw <tracker_home> - -To test the mail gateway on unix systems, try:: - - echo test |mail -s '[issue] test' issue_tracker@your.domain - - -Setup 2: As a regular cron job using a mailbox source -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Set ``roundup-mailgw`` up to run every 10 minutes or so. For example:: - - 10 * * * * /usr/local/bin/roundup-mailgw <tracker_home> mailbox <mail_spool_file> - -Where the ``mail_spool_file`` argument is the location of the roundup submission -user's mail spool. On most systems, the spool for a user "issue_tracker" -will be "``/var/mail/issue_tracker``". - -Setup 3: As a regular cron job using a POP source -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -To retrieve from a POP mailbox, use a similar cron entry to the mailbox one:: - - 10 * * * * /usr/local/bin/roundup-mailgw <tracker_home> pop <pop_spec> - -where pop_spec is "``username:password@server``" that specifies the roundup -submission user's POP account name, password and server. - - -Web Interface -------------- - -This software will work through apache or stand-alone. - -Stand-alone: - 1. Edit roundup-server at the top - ``TRACKER_HOMES`` needs to know - about your tracker. You may also specify the values for - ``TRACKER_HOMES`` on the command-line using "name=home" pairs. - - 2. "``roundup-server [-p port] (name=tracker_home)*``" (hostname may be "") - - 3. Load up the page "``/<tracker name>/index``" where tracker name is the name - you nominated in ``TRACKER_HOMES``. - -Apache: - 1. The CGI script is found in the cgi-bin directory of the roundup - distribution. - - 2. Make sure roundup.cgi is executable. Edit it at the top - - ``TRACKER_HOMES`` needs to know about your tracker. - - 3. Edit your "``/etc/httpd/conf/httpd.conf``" and make sure that the - "``/home/httpd/html/roundup/roundup.cgi``" script will be treated as a CGI script. - - 4. Re-start your apache to re-load the config if necessary. - - 5. Load up the page "``/roundup/roundup.cgi/index/``" where tracker name is the - name you nominated in ``TRACKER_HOMES``. - - 6. To use the CGI script unchanged, which allows much easier updates, add - these directives to your "httpd.conf":: - - SetEnv ROUNDUP_LOG "/var/log/roundup.log" - SetEnv TRACKER_HOMES "Default=/usr/local/share/roundup/trackers/Default" - SetEnv ROUNDUP_DEBUG "0" - - 7. On Windows, write a batch file "roundup.bat" similar to the one below and - place it into your cgi-bin directory:: - - @echo off - set ROUNDUP_LOG=c:\Python21\share\roundup\cgi.log - set TRACKER_HOMES=Default=c:\Python21\share\roundup\trackers\Default; - set ROUNDUP_DEBUG=0 - c:\Python21\python.exe c:\Python21\share\roundup\cgi-bin\roundup.cgi - - -Users ------ - -Users and permissions -~~~~~~~~~~~~~~~~~~~~~ - -By default, roundup automatically creates one user when the tracker database is -initialised (using roundup-admin init). The user is "admin" and the password is -the one you supply at that time. - -If users attempt to use roundup in any manner and are not identified to roundup, -they will be using the database in a read-only mode. That is, if roundup doesn't -know who they are, they can't change anything. This has the following -repurcussions: - -Command-line interface - The data modification commands (create, init, retire, set) are performed - as the "admin" user. It is therefore important that the database be - protected by the filesystem if protection is required. On a Unix system, - the easiest and most flexible method of doing so is: - - 1. Add a new user and group to your system (e.g. "issue_tracker") - - 2. When creating a new tracker home, use the following commands - (substituting tracker_home for the directory you want to use):: - - mkdir tracker_home - chown issue_tracker:issue_tracker tracker_home - chmod g+rwxs tracker_home - roundup-admin -i tracker_home init - - 3. Now, edit the /etc/group line for the issue_tracker group so it - includes the unix logins of all the users who are going to - administer your roundup tracker. If you're running the web or mail - gateways, then be sure to include those users in the group too (on - some Linux systems, these users are "www" or "apache" and "mail".) - -E-Mail interface - Users are identified by e-mail address - a new user entry will be created - for any e-mail address that is not recognised, so users are always - identified by roundup. - -Web interface - Unidentified users have read-only access. If the users database has an - entry with the username "anonymous", then unidentified users are - automatically logged in as that user. This gives them write access. - -**anonymous access and the ANONYMOUS_* configurations.** - - -Adding users -~~~~~~~~~~~~ - -To add users, use one of the following interfaces: - -1. On the web, access the URL .../<tracker name>/newuser to bring up a form - which may be used to add a new user. - -2. On the command-line, use:: - - roundup-admin -i <tracker home> create user username=bozo password=bozo - address=richard@clown.org - - Supply the admin username and password. roundup-admin will print the id - of the new user. - -3. Any e-mail sent to roundup from an address that doesn't match an existing - user in the database will result in a new user entry being created for - that user. - - -Issues ------- - -To add issues, use one of the following interfaces: - -1. On the web, access the URL .../<tracker name>/newissue to bring up a - form which may be used to add a new issue. - -2. On the command-line, use:: - - roundup-admin -i <tracker home> create issue title="test issue" - - Supply the admin username and password. roundup-admin will print the id - of the new issue. - -3. Any e-mail sent to roundup with the subject line containing [issue] will - automatically created a new issue in the database using the contents of - the e-mail. - ------------------ - -Back to `Table of Contents`_ - -Next: `User Guide`_ - -.. _`table of contents`: index.html -.. _`user guide`: user_guide.html -.. _`customisation`: customizing.html -
--- a/doc/index.txt Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,82 +0,0 @@ -======================================================= -Roundup: an Issue-Tracking System for Knowledge Workers -======================================================= - -Contents -======== - -- Overview_ and Features_ -- Installation_ and `Getting Started`_ (and Upgrading_) -- `User Guide`_ -- `Customising Roundup`_ -- `Maintaining Roundup Trackers`_ -- `Roundup's Design`_ (original_) -- `Developing Roundup`_ -- Contact_ -- Acknowledgements_ -- License_ - -.. _Features: features.html -.. _Overview: overview.html -.. _Installation: installation.html -.. _`Getting Started`: getting_started.html -.. _`User Guide`: user_guide.html -.. _`Customising Roundup`: customizing.html -.. _`Maintaining Roundup Trackers`: maintenance.html -.. _`Developing Roundup`: developers.html -.. _`Roundup's Design`: design.html -.. _original: spec.html -.. _Upgrading: upgrading.html - - -Contact -======= - -For general support enquiries about usage, a mailing list is available: - - roundup-users@sourceforge.net - -If you've got a great idea for roundup, or have found a bug, please -submit -an issue to the tracker at: - - http://sourceforge.net/tracker/?group_id=31577 - -For discussions about developing or enhancing roundup: - - roundup-devel@sourceforge.net - -The admin for this project is Richard Jones: - - richard@users.sourceforge.net - -but he should only be contacted directly when none of the -above avenues of contact are suitable. - - -Acknowledgements -================ - -Go Ping, you rock! Also, go Bizar Software and ekit.com for letting me -implement this system on their time. - -Thanks also to the many people on the mailing list and in the sourceforge -project: -Anthony Baxter, -Jeff Blaine, -Duncan Booth, -Titus Brown, -Roch'e Compaan, -Engelbert Gruber, -Juergen Hermann, -Gordon McMillan, -Patrick Ohly, -Dougal Scott, -Stefan Seefeld. - - -License -======= - -See COPYING.txt in the software distribution for the licensing terms. -
--- a/doc/installation.txt Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,408 +0,0 @@ -================== -Installing Roundup -================== - -:Version: $Revision: 1.28 $ - -.. contents:: - - -Overview -======== - -Broken out separately, there are several conceptual pieces to a -Roundup installation: - -Roundup trackers - Trackers consist of issues (be they bug reports or otherwise), tracker - configuration file(s), web HTML files etc. Roundup trackers are initialised - with a "Template" which defines the fields usable/assignable on a - per-issue basis. Descriptions of the provided templates are given in - `choosing your template`_. - -Roundup support code - Installed into your Python install's lib directory - -Roundup scripts - These include the email gateway, the roundup - HTTP server, the roundup administration command-line interface, etc. - - -Prerequisites -============= - -Python 2.1.1 or newer with a functioning anydbm or bsddb module. Download the -latest version from http://www.python.org/. It is highly recommended that -users install the latest patch version of python - 2.1.3 or 2.2.1 - as these -contain many fixes to serious bugs. - -If you want to use Berkeley DB bsddb3 with Roundup, use version 3.3.0 or -later. Download the latest version from http://pybsddb.sourceforge.net/. - - -Getting Roundup -=============== - -Download the latest version from http://roundup.sf.net/. - -Testing your Python -------------------- - -Once you've unpacked roundup's source, run ``python ./run_tests`` in the -source directory and make sure there are no errors. -If there are errors, please let us know! - -If the above fails, you may be using the wrong version of python. Try -``python2 ./run_tests``. If that works, you will need to substitute -``python2`` for ``python`` in all further commands you use in relation to -Roundup -- from installation and scripts. - - -Installation -============ - -Set aside 15-30 minutes. Please make sure you're using a supported version of -Python -- see `testing your python`_. There's four steps to follow in your -installation: - -1. `basic installation steps`_ that all installers must follow -2. then optionally `configure a web interface`_ -3. and optionally `configure an email interface`_ -4. `shared environment steps`_ to take if you're installing on a shared - UNIX machine and want to restrict local access to roundup - -Most users will only need to follow the first step, since the environment will -be a trusted one. - - -Basic Installation Steps ------------------------- - -1. To install the Roundup support code into your Python tree and - Roundup scripts into /usr/local/bin:: - - python setup.py install - - If you would like to place the Roundup scripts in a directory other - than ``/usr/local/bin``, then specify the preferred location with - ``--install-script``. For example, to install them in - ``/opt/roundup/bin``:: - - python setup.py install --install-scripts=/opt/roundup/bin - -2. To create a Roundup tracker (necessary to do before you can - use the software in any real fashion): - - a. (Optional) If you intend to keep your roundup trackers - under one top level directory which does not exist yet, - you should create that directory now. Example:: - - mkdir /opt/roundup/trackers - - b. Either add the Roundup script location to your ``PATH`` - environment variable or specify the full path to - the command in the next step. - - c. Install a new tracker with the command ``roundup-admin install``. - You will be asked a series of questions. Descriptions of the provided - templates can be found in `choosing your template`_ below. Descriptions - of the available backends can be found in `choosing your backend`_ - below. The questions will be something like (you may have more - templates or backends available):: - - Enter tracker home: /opt/roundup/trackers/support - Templates: classic - Select template [classic]: classic - Back ends: anydbm, bsddb - Select backend [anydbm]: anydbm - - You will now be directed to edit the tracker configuration and - initial schema. See `Customising Roundup`_ for details on configuration - and schema changes. Note that you may change any of the configuration - after you've initialised the tracker - it's just better to have valid - values for this stuff now. - - d. Initialise the tracker database with ``roundup-admin initialise``. - You will need to supply an admin password at this step. You will be - prompted:: - - Admin Password: - Confirm: - - Once this is done, the tracker has been created. - -At this point, your tracker is set up, but doesn't have a nice user interface. -To set that up, we need to `configure a web interface`_ and optionally -`configure an email interface`_. - - -Choosing Your Template ----------------------- - -Classic Template -~~~~~~~~~~~~~~~~ - -The classic template is the one defined in the `Roundup Specification`_. It -holds issues which have priorities and statuses. Each issue may also have a -set of messages which are disseminated to the issue's list of nosy users. - -Minimal Template -~~~~~~~~~~~~~~~~ - -The minimal template has the minimum setup required for a tracker -installation. That is, it has the configuration files, defines a user database -and the basic HTML interface to that. It's a completely clean slate for you to -create your tracker on. - - -Choosing Your Backend ---------------------- - -The actual storage of Roundup tracker information is handled by backends. -There's several to choose from, each with benefits and limitations: - -**anydbm** - This backend is guaranteed to work on any system that Python runs on. It - will generally choose the best dbm backend that is available on your system - (from the list dbhash, gdbm, dbm, dumbdbm). It is the least scaleable of all - backends, but performs well enough for a smallish tracker (a couple of - thousand issues, under fifty users, ...). -**bsddb** - This effectively the same as anydbm, but uses the bsddb backend. This allows - it to gain some performance and scaling benefits. -**bsddb3** - Again, this effectively the same as anydbm, but uses the bsddb3 backend. - This allows it to gain some performance and scaling benefits. -**sqlite_** - This uses the SQLite_ embedded RDBMS to provide a fast, scaleable backend. - There are no limitations, and it's much faster and more scaleable than the - dbm backends. -**metakit_** - This backend is implemented over the metakit_ storage system, using Mk4Py as - the interface. It scales much better than the dbm backends. -**gadfly** - This is a proof-of-concept relational database backend, not really intended - for actual production use, although it can be. It uses the Gadfly RDBMS - to store data. It is unable to perform string searches due to gadfly not - having a LIKE operation. It should scale well, assuming a client/server - setup is used. It's much slower than even the dbm backends. - -Note: you may set your tracker up with the anydbm backend (which is guaranteed -to be available) and switch to one of the other backends at any time using the -instructions in the `maintenance documentation`_. - - -Configure a Web Interface -------------------------- - -There are three web interfaces to choose from: - -1. `web server cgi-bin`_ -2. `stand-alone web server`_ -3. `Zope product - ZRoundup`_ - -You may need to give the web server user permission to access the tracker home -- see the `shared environment steps`_ for information. - - -Web Server cgi-bin -~~~~~~~~~~~~~~~~~~ - -A benefit of using the cgi-bin approach is that it's the easiest way to -restrict access to your tracker to only use HTTPS. Access will be slower -than through the `stand-alone web server`_ though. - -Copy the ``cgi-bin/roundup.cgi`` file to your web server's ``cgi-bin`` -directory. You will need to configure it to tell it where your tracker home -is. You can do this either: - -through an environment variable - set the variable TRACKER_HOMES to be a colon (":") separated list of - name=home pairs (if you're using apache, the SetEnv directive can do this) -directly in the ``roundup.cgi`` file itself - add your instance to the TRACKER_HOMES variable as ``'name': 'home'`` - -The "name" part of the configuration will appear in the URL and identifies the -tracker (so you may have more than one tracker per cgi-bin script). Make sure -there are no spaces or other illegal characters in it (to be safe, stick to -letters and numbers). The "name" forms part of the URL that appears in the -tracker config TRACKER_WEB variable, so make sure they match. The "home" -part of the configuration is the tracker home directory. - -Stand-alone Web Server -~~~~~~~~~~~~~~~~~~~~~~ - -This approach will give you the fastest of the three web interfaces. You may -investigate using ProxyPass or similar configuration in apache to have your -tracker accessed through the same URL as other systems. - -The stand-alone web server is started with the command ``roundup-server``. It -has several options - display them with ``roundup-server -h``. - -The tracker home configuration is similar to the cgi-bin - you may either edit -the script to change the TRACKER_HOMES variable or you may supply the -name=home values on the command-line after all the other options. - -To make the server run in the background, use the "-d" option, specifying the -name of a file to write the server process id (pid) to. - - -Zope Product - ZRoundup -~~~~~~~~~~~~~~~~~~~~~~~ - -ZRoundup installs as a regular Zope product. Copy the ZRoundup directory to -your Products directory either in INSTANCE_HOME/Products or the Zope -code tree lib/python/Products. - -When you next (re)start up Zope, you will be able to add a ZRoundup object -that interfaces to your new tracker. - - -Configure an Email Interface ----------------------------- - -If you don't want to use the email component of Roundup, then remove the -"``nosyreator.py``" module from your tracker "``detectors``" directory. - -There are three supported ways to get emailed issues into the -Roundup tracker. You should pick ONE of the following, all -of which will continue my example setup from above: - -As a mail alias pipe process -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Set up a mail alias called "issue_tracker" as (include the quote marks): -"``|/usr/bin/python /usr/local/bin/roundup-mailgw <tracker_home>``" - -In some installations (e.g. RedHat 6.2 I think) you'll need to set up smrsh so -sendmail will accept the pipe command. In that case, symlink -``/etc/smrsh/roundup-mailgw`` to "``/usr/local/bin/roundup-mailgw``" and change -the command to:: - - |roundup-mailgw /opt/roundup/trackers/support - -To test the mail gateway on unix systems, try:: - - echo test |mail -s '[issue] test' support@YOUR_DOMAIN_HERE - -As a regular cron job using a mailbox source -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Set ``roundup-mailgw`` up to run every 10 minutes or so. For example:: - - 10 * * * * /usr/local/bin/roundup-mailgw /opt/roundup/trackers/support mailbox <mail_spool_file> - -Where the ``mail_spool_file`` argument is the location of the roundup submission -user's mail spool. On most systems, the spool for a user "issue_tracker" -will be "``/var/mail/issue_tracker``". - -As a regular cron job using a POP source -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -To retrieve from a POP mailbox, use a similar cron entry to the mailbox one:: - - 10 * * * * /usr/local/bin/roundup-mailgw /opt/roundup/trackers/support pop <pop_spec> - -where pop_spec is "``username:password@server``" that specifies the roundup -submission user's POP account name, password and server. - - -Shared Environment Steps ------------------------- - -Each tracker ideally should have its own UNIX group, so create -a UNIX group (edit ``/etc/group`` or your appropriate NIS map if -you're using NIS). To continue with my examples so far, I would -create the UNIX group 'support', although the name of the UNIX -group does not have to be the same as the tracker name. To this -'support' group I then add all of the UNIX usernames who will be -working with this Roundup tracker. In addition to 'real' users, -the Roundup email gateway will need to have permissions to this -area as well, so add the user your mail service runs as to the -group. The UNIX group might then look like:: - - support:*:1002:jblaine,samh,geezer,mail - -If you intend to use the web interface (as most people do), you -should also add the username your web server runs as to the group. -My group now looks like this:: - - support:*:1002:jblaine,samh,geezer,mail,apache - -The tracker "db" directory should be chmod'ed g+sw so that the group can -write to the database, and any new files created in the database will be owned -by the group. - -An alternative to the above is to create a new user who has the sole -responsibility of running roundup. This user: - -1. runs the CGI interface daemon -2. runs regular polls for email -3. runs regular checks (using cron) to ensure the daemon is up -4. optionally has no login password so that nobody but the "root" user - may actually login and play with the roundup setup. - - -Maintenance -=========== - -Read the separate `maintenance documentation`_ for information about how to -perform common maintenance tasks with Roundup. - - -Upgrading -========= - -Read the separate `upgrading document`_, which describes the steps needed to -upgrade existing tracker trackers for each version of Roundup that is -released. - - -Further Reading -=============== - -If you intend to use Roundup with anything other than the defualt -templates, if you would like to hack on Roundup, or if you would -like implementation details, you should read `Customising Roundup`_. - - -Platform-Specific Notes -======================= - -Sendmail smrsh --------------- - -If you use Sendmail's ``smrsh`` mechanism, you will need to tell -smrsh that roundup-mailgw is a valid/trusted mail handler -before it will work. - -This is usually done via the following 2 steps: - -1. make a symlink in ``/etc/smrsh`` called ``roundup-mailgw`` - which points to the full path of your actual ``roundup-mailgw`` - script. - -2. change your alias to ``"|roundup-mailgw <tracker_home>"`` - - -Linux ------ - -Python 2.1.1 as shipped with SuSE7.3 might be missing module -``_weakref``. - -------------------------------------------------------------------------------- - -Back to `Table of Contents`_ - -Next: `Getting Started`_ - -.. _`table of contents`: index.html -.. _`getting started`: getting_started.html -.. _`roundup specification`: spec.html -.. _`customising roundup`: customizing.html -.. _`upgrading document`: upgrading.html -.. _`maintenance documentation`: maintenance.html -.. _sqlite: http://www.hwaci.com/sw/sqlite/ -.. _metakit: http://www.equi4.com/metakit/ -
--- a/doc/upgrading.txt Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,729 +0,0 @@ -====================================== -Upgrading to newer versions of Roundup -====================================== - -Please read each section carefully and edit your tracker home files -accordingly. - -.. contents:: - -Migrating from 0.4.x to 0.5.0 -============================= - -This has been a fairly major revision of Roundup: - -1. Brand new, much more powerful, flexible, tasty and nutritious templating. - Unfortunately, this means all your current templates are useless. Hopefully - the new documentation and examples will be enough to help you make the - transition. Please don't hesitate to ask on roundup-users for help (or - complete conversions if you're completely stuck)! -2. The database backed got a lot more flexible, allowing Metakit and SQL - databases! The only decent SQL database implemented at present is sqlite, - but others shouldn't be a whole lot more work. -3. A brand new, highly flexible and much more robust security system including - a system of Permissions, Roles and Role assignments to users. You may now - define your own Permissions that may be checked in CGI transactions. -4. Journalling has been made less storage-hungry, so has been turned on - by default *except* for author, recipient and nosy link/unlink events. You - are advised to turn it off in your trackers too. -5. We've changed the terminology from "instance" to "tracker", to ease the - learning curve/impact for new users. -6. Because of the above changes, the tracker configuration has seen some - major changes. See below for the details. - -Please, **back up your database** before you start the migration process. This -is as simple as copying the "db" directory and all its contents from your -tracker to somewhere safe. - - -0.5.0 Configuration -------------------- - -First up, rename your ``instance_config.py`` file to just ``config.py``. - -Then edit your tracker's ``__init__.py`` module. It'll currently look -like this:: - - from instance_config import * - try: - from dbinit import * - except ImportError: - pass # in installdir (probably :) - from interfaces import * - -and it needs to be:: - - import config - from dbinit import open, init - from interfaces import Client, MailGW - -Due to the new templating having a top-level ``page`` that defines links for -searching, indexes, adding items etc, the following variables are no longer -used: - -- HEADER_INDEX_LINKS -- HEADER_ADD_LINKS -- HEADER_SEARCH_LINKS -- SEARCH_FILTERS -- DEFAULT_INDEX -- UNASSIGNED_INDEX -- USER_INDEX -- ISSUE_FILTER - -The new security implementation will require additions to the dbinit module, -but also removes the need for the following tracker config variables: - -- ANONYMOUS_ACCESS -- ANONYMOUS_REGISTER - -but requires two new variables which define the Roles assigned to users who -register through the web and e-mail interfaces: - -- NEW_WEB_USER_ROLES -- NEW_EMAIL_USER_ROLES - -in both cases, 'User' is a good initial setting. To emulate -``ANONYMOUS_ACCESS='deny'``, remove all "View" Permissions from the -"Anonymous" Role. To emulate ``ANONYMOUS_REGISTER='deny'``, remove the "Web -Registration" and/or the "Email Registration" Permission from the "Anonymous" -Role. See the section on customising security in the `customisation -documentation`_ for more information. - -Finally, the following config variables have been renamed to make more sense: - -- INSTANCE_HOME -> TRACKER_HOME -- INSTANCE_NAME -> TRACKER_NAME -- ISSUE_TRACKER_WEB -> TRACKER_WEB -- ISSUE_TRACKER_EMAIL -> TRACKER_EMAIL - - -0.5.0 Schema Specification --------------------------- - -0.5.0 Database backend changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Your select_db module in your tracker has changed a fair bit. Where it used -to contain:: - - # WARNING: DO NOT EDIT THIS FILE!!! - from roundup.backends.back_anydbm import Database - -it must now contain:: - - # WARNING: DO NOT EDIT THIS FILE!!! - from roundup.backends.back_anydbm import Database, Class, FileClass, IssueClass - -Yes, I realise the irony of the "DO NOT EDIT THIS FILE" statement :) -Note the addition of the Class, FileClass, IssueClass imports. These are very -important, as they're going to make the next change work too. You now need to -modify the top of the dbinit module in your tracker from:: - - import instance_config - from roundup import roundupdb - from select_db import Database - - from roundup.roundupdb import Class, FileClass - - class Database(roundupdb.Database, select_db.Database): - ''' Creates a hybrid database from: - . the selected database back-end from select_db - . the roundup extensions from roundupdb - ''' - pass - - class IssueClass(roundupdb.IssueClass): - ''' issues need the email information - ''' - pass - -to:: - - import config - from select_db import Database, Class, FileClass, IssueClass - -Yes, remove the Database and IssueClass definitions and those other imports. -They're not needed any more! - -Look for places in dbinit.py where ``instance_config`` is used too, and -rename them ``config``. - - -0.5.0 Journalling changes -~~~~~~~~~~~~~~~~~~~~~~~~~ - -Journalling has been optimised for storage. Journalling of links has been -turned back on by default. If your tracker has a large user base, you may wish -to turn off journalling of nosy list, message author and message recipient -link and unlink events. You do this by adding ``do_journal='no'`` to the Class -initialisation in your dbinit. For example, your *msg* class initialisation -probably looks like this:: - - msg = FileClass(db, "msg", - author=Link("user"), recipients=Multilink("user"), - date=Date(), summary=String(), - files=Multilink("file"), - messageid=String(), inreplyto=String()) - -to turn off journalling of author and recipient link events, add -``do_journal='no'`` to the ``author=Link("user")`` part of the statement, -like so:: - - msg = FileClass(db, "msg", - author=Link("user", do_journal='no'), - recipients=Multilink("user", do_journal='no'), - date=Date(), summary=String(), - files=Multilink("file"), - messageid=String(), inreplyto=String()) - -Nosy list link event journalling is actually turned off by default now. If you -want to turn it on, change to your issue class' nosy list, change its -definition from:: - - issue = IssueClass(db, "issue", - assignedto=Link("user"), topic=Multilink("keyword"), - priority=Link("priority"), status=Link("status")) - -to:: - - issue = IssueClass(db, "issue", nosy=Multilink("user", do_journal='yes'), - assignedto=Link("user"), topic=Multilink("keyword"), - priority=Link("priority"), status=Link("status")) - -noting that your definition of the nosy Multilink will override the normal one. - - -0.5.0 User schema changes -~~~~~~~~~~~~~~~~~~~~~~~~~ - -Users have two more properties, "queries" and "roles". You'll have something -like this in your dbinit module now:: - - user = Class(db, "user", - username=String(), password=Password(), - address=String(), realname=String(), - phone=String(), organisation=String(), - alternate_addresses=String()) - user.setkey("username") - -and you'll need to add the new properties and the new "query" class to it -like so:: - - query = Class(db, "query", - klass=String(), name=String(), - url=String()) - query.setkey("name") - - # Note: roles is a comma-separated string of Role names - user = Class(db, "user", - username=String(), password=Password(), - address=String(), realname=String(), - phone=String(), organisation=String(), - alternate_addresses=String(), - queries=Multilink('query'), roles=String()) - user.setkey("username") - -The "queries" property is used to store off the user's favourite database -queries. The "roles" property is explained below in `0.5.0 Security -Settings`_. - - -0.5.0 Security Settings -~~~~~~~~~~~~~~~~~~~~~~~ - -See the `security documentation`_ for an explanation of how the new security -system works. In a nutshell though, the security is handled as a four step -process: - -1. Permissions are defined as having a name and optionally a hyperdb class - they're specific to, -2. Roles are defined that have one or more Permissions, -3. Users are assigned Roles in their "roles" property, and finally -4. Roundup checks that users have appropriate Permissions at appropriate times - (like editing issues). - -Your tracker dbinit module's *open* function now has to define any -Permissions that are specific to your tracker, and also the assignment -of Permissions to Roles. At the moment, your open function -ends with:: - - import detectors - detectors.init(db) - - return db - -and what we need to do is insert some commands that will set up the security -parameters. Right above the ``import detectors`` line, you'll want to insert -these lines:: - - # - # SECURITY SETTINGS - # - # new permissions for this schema - for cl in 'issue', 'file', 'msg', 'user': - db.security.addPermission(name="Edit", klass=cl, - description="User is allowed to edit "+cl) - db.security.addPermission(name="View", klass=cl, - description="User is allowed to access "+cl) - - # Assign the access and edit permissions for issue, file and message - # to regular users now - for cl in 'issue', 'file', 'msg': - p = db.security.getPermission('View', cl) - db.security.addPermissionToRole('User', p) - p = db.security.getPermission('Edit', cl) - db.security.addPermissionToRole('User', p) - # and give the regular users access to the web and email interface - p = db.security.getPermission('Web Access') - db.security.addPermissionToRole('User', p) - p = db.security.getPermission('Email Access') - db.security.addPermissionToRole('User', p) - - # May users view other user information? Comment these lines out - # if you don't want them to - p = db.security.getPermission('View', 'user') - db.security.addPermissionToRole('User', p) - - # Assign the appropriate permissions to the anonymous user's Anonymous - # Role. Choices here are: - # - Allow anonymous users to register through the web - p = db.security.getPermission('Web Registration') - db.security.addPermissionToRole('Anonymous', p) - # - Allow anonymous (new) users to register through the email gateway - p = db.security.getPermission('Email Registration') - db.security.addPermissionToRole('Anonymous', p) - # - Allow anonymous users access to the "issue" class of data - # Note: this also grants access to related information like files, - # messages, statuses etc that are linked to issues - #p = db.security.getPermission('View', 'issue') - #db.security.addPermissionToRole('Anonymous', p) - # - Allow anonymous users access to edit the "issue" class of data - # Note: this also grants access to create related information like - # files and messages etc that are linked to issues - #p = db.security.getPermission('Edit', 'issue') - #db.security.addPermissionToRole('Anonymous', p) - - # oh, g'wan, let anonymous access the web interface too - p = db.security.getPermission('Web Access') - db.security.addPermissionToRole('Anonymous', p) - -Note in the comments there the places where you might change the permissions -to restrict users or grant users more access. If you've created additional -classes that users should be able to edit and view, then you should add them -to the "new permissions for this schema" section at the start of the security -block. Then add them to the "Assign the access and edit permissions" section -too, so people actually have the new Permission you've created. - -One final change is needed that finishes off the security system's -initialisation. We need to add a call to ``db.post_init()`` at the end of the -dbinit open() function. Add it like this:: - - import detectors - detectors.init(db) - - # schema is set up - run any post-initialisation - db.post_init() - return db - -You may verify the setup of Permissions and Roles using the new -"``roundup-admin security``" command. - - -0.5.0 User changes -~~~~~~~~~~~~~~~~~~ - -To support all those schema changes, you'll need to massage your user database -a little too, to: - -1. make sure there's an "anonymous" user - this user is mandatory now and is - the one that unknown users are logged in as. -2. make sure all users have at least one Role. - -If you don't have the "anonymous" user, create it now with the command:: - - roundup-admin create user username=anonymous roles=Anonymous - -making sure the capitalisation is the same as above. Once you've done that, -you'll need to set the roles property on all users to a reasonable default. -The admin user should get "Admin", the anonymous user "Anonymous" -and all other users "User". The ``fixroles.py`` script in the tools directory -will do this. Run it like so (where python is your python 2+ binary):: - - python tools/fixroles.py -i <tracker home> - - - -0.5.0 CGI interface changes ---------------------------- - -The CGI interface code was completely reorganised and largely rewritten. The -end result is that this section of your tracker interfaces module will need -changing from:: - - from roundup import mailgw - from roundup.cgi import client - - class Client(client.Client): - ''' derives basic CGI implementation from the standard module, - with any specific extensions - ''' - pass - -to:: - - from roundup import cgi_client, mailgw - from roundup.i18n import _ - - class Client(cgi_client.Client): - ''' derives basic CGI implementation from the standard module, - with any specific extensions - ''' - pass - -You will also need to install the new version of roundup.cgi from the source -cgi-bin directory if you're using it. - - -0.5.0 HTML templating ---------------------- - -You'll want to make a backup of your current tracker html directory. You -should then copy the html directory from the Roundup source "classic" template -and modify it according to your local schema changes. - -If you need help with the new templating system, please ask questions on the -roundup-users mailing list (available through the roundup project page on -sourceforge, http://roundup.sf.net/) - - -0.5.0 Detectors ---------------- - -The nosy reactor has been updated to handle the tracker not having an -"assignedto" property on issues. You may want to copy it into your tracker's -detectors directory. Chances are you've already fixed it though :) - - -Migrating from 0.4.1 to 0.4.2 -============================= - -0.4.2 Configuration -------------------- -The USER_INDEX definition introduced in 0.4.1 was too restrictive in its -allowing replacement of 'assignedto' with the user's userid. Users must change -the None value of 'assignedto' to 'CURRENT USER' (the string, in quotes) for -the replacement behaviour to occur now. - -The new configuration variables are: - -- EMAIL_KEEP_QUOTED_TEXT -- EMAIL_LEAVE_BODY_UNCHANGED -- ADD_RECIPIENTS_TO_NOSY - -See the sample configuration files in:: - - <roundup source>/roundup/templates/classic/instance_config.py - -and:: - - <roundup source>/roundup/templates/extended/instance_config.py - -and the `customisation documentation`_ for information on how they're used. - - -0.4.2 Changes to detectors --------------------------- -You will need to copy the detectors from the distribution into your instance -home "detectors" directory. If you used the classic schema, the detectors -are in:: - - <roundup source>/roundup/templates/classic/detectors/ - -If you used the extended schema, the detectors are in:: - - <roundup source>/roundup/templates/extended/detectors/ - -The change means that schema-specific code has been removed from the -mail gateway and cgi interface and made into auditors: - -- nosyreactor.py has now got an updatenosy auditor which updates the nosy - list with author, recipient and assignedto information. -- statusauditor.py makes the unread or resolved -> chatting changes and - presets the status of an issue to unread. - -There's also a bug or two fixed in the nosyreactor code. - -0.4.2 HTML templating changes ------------------------------ -The link() htmltemplate function now has a "showid" option for links and -multilinks. When true, it only displays the linked item id as the anchor -text. The link value is displayed as a tooltip using the title anchor -attribute. To use in eg. the superseder field, have something like this:: - - <td> - <display call="field('superseder', showid=1)"> - <display call="classhelp('issue', 'id,title', label='list', width=500)"> - <property name="superseder"> - <br>View: <display call="link('superseder', showid=1)"> - </property> - </td> - -The stylesheets have been cleaned up too. You may want to use the newer -versions in:: - - <roundup source>/roundup/templates/<template>/html/default.css - - - -Migrating from 0.4.0 to 0.4.1 -============================= - -0.4.1 Files storage -------------------- - -Messages and files from newly created issues will be put into subdierectories -in thousands e.g. msg123 will be put into files/msg/0/msg123, file2003 -will go into files/file/2/file2003. Previous messages are still found, but -could be put into this structure. - -0.4.1 Configuration -------------------- - -To allow more fine-grained access control, the variable used to check -permission to auto-register users in the mail gateway is now called -ANONYMOUS_REGISTER_MAIL rather than overloading ANONYMOUS_REGISTER. If the -variable doesn't exist, then ANONYMOUS_REGISTER is tested as before. - -Configuring the links in the web header is now easier too. The following -variables have been added to the classic instance_config.py:: - - HEADER_INDEX_LINKS - defines the "index" links to be made available - HEADER_ADD_LINKS - defines the "add" links - DEFAULT_INDEX - specifies the index view for DEFAULT - UNASSIGNED_INDEX - specifies the index view for UNASSIGNED - USER_INDEX - specifies the index view for USER - -See the <roundup source>/roundup/templates/classic/instance_config.py for more -information - including how the variables are to be set up. Most users will -just be able to copy the variables from the source to their instance home. If -you've modified the header by changing the source of the interfaces.py file in -the instance home, you'll need to remove that customisation and move it into -the appropriate variables in instance_config.py. - -The extended schema has similar variables added too - see the source for more -info. - -0.4.1 Alternate E-Mail Addresses --------------------------------- - -If you add the property "alternate_addresses" to your user class, your users -will be able to register alternate email addresses that they may use to -communicate with roundup as. All email from roundup will continue to be sent -to their primary address. - -If you have not edited the dbinit.py file in your instance home directory, -you may simply copy the new dbinit.py file from the core code. If you used -the classic schema, the interfaces file is in:: - - <roundup source>/roundup/templates/classic/dbinit.py - -If you used the extended schema, the file is in:: - - <roundup source>/roundup/templates/extended/dbinit.py - -If you have modified your dbinit.py file, you need to edit the dbinit.py -file in your instance home directory. Find the lines which define the user -class:: - - user = Class(db, "msg", - username=String(), password=Password(), - address=String(), realname=String(), - phone=String(), organisation=String(), - alternate_addresses=String()) - -You will also want to add the property to the user's details page. The -template for this is the "user.item" file in your instance home "html" -directory. Similar to above, you may copy the file from the roundup source if -you haven't modified it. Otherwise, add the following to the template:: - - <display call="multiline('alternate_addresses')"> - -with appropriate labelling etc. See the standard template for an idea. - - - -Migrating from 0.3.x to 0.4.0 -============================= - -0.4.0 Message-ID and In-Reply-To addition ------------------------------------------ -0.4.0 adds the tracking of messages by message-id and allows threading -using in-reply-to. Most e-mail clients support threading using this -feature, and we hope to add support for it to the web gateway. If you -have not edited the dbinit.py file in your instance home directory, you may -simply copy the new dbinit.py file from the core code. If you used the -classic schema, the interfaces file is in:: - - <roundup source>/roundup/templates/classic/dbinit.py - -If you used the extended schema, the file is in:: - - <roundup source>/roundup/templates/extended/dbinit.py - -If you have modified your dbinit.py file, you need to edit the dbinit.py -file in your instance home directory. Find the lines which define the msg -class:: - - msg = FileClass(db, "msg", - author=Link("user"), recipients=Multilink("user"), - date=Date(), summary=String(), - files=Multilink("file")) - -and add the messageid and inreplyto properties like so:: - - msg = FileClass(db, "msg", - author=Link("user"), recipients=Multilink("user"), - date=Date(), summary=String(), - files=Multilink("file"), - messageid=String(), inreplyto=String()) - -Also, configuration is being cleaned up. This means that your dbinit.py will -also need to be changed in the open function. If you haven't changed your -dbinit.py, the above copy will be enough. If you have, you'll need to change -the line (round line 50):: - - db = Database(instance_config.DATABASE, name) - -to:: - - db = Database(instance_config, name) - - -0.4.0 Configuration --------------------- -``TRACKER_NAME`` and ``EMAIL_SIGNATURE_POSITION`` have been added to the -instance_config.py. The simplest solution is to copy the default values -from template in the core source. - -The mail gateway now checks ``ANONYMOUS_REGISTER`` to see if unknown users -are to be automatically registered with the tracker. If it is set to "deny" -then unknown users will not have access. If it is set to "allow" they will be -automatically registered with the tracker. - - -0.4.0 CGI script roundup.cgi ----------------------------- -The CGI script has been updated with some features and a bugfix, so you should -copy it from the roundup cgi-bin source directory again. Make sure you update -the ROUNDUP_INSTANCE_HOMES after the copy. - - -0.4.0 Nosy reactor ------------------- -The nosy reactor has also changed - copy the nosyreactor.py file from the core -source:: - - <roundup source>/roundup/templates/<template>/detectors/nosyreactor.py - -to your instance home "detectors" directory. - - -0.4.0 HTML templating ---------------------- -The field() function was incorrectly implemented - links and multilinks now -display as text fields when rendered using field(). To display a menu (drop- -down or select box) you need to use the menu() function. - - - -Migrating from 0.2.x to 0.3.x -============================= - -0.3.x Cookie Authentication changes ------------------------------------ -0.3.0 introduces cookie authentication - you will need to copy the -interfaces.py file from the roundup source to your instance home to enable -authentication. If you used the classic schema, the interfaces file is in:: - - <roundup source>/roundup/templates/classic/interfaces.py - -If you used the extended schema, the file is in:: - - <roundup source>/roundup/templates/extended/interfaces.py - -If you have modified your interfaces.Client class, you will need to take -note of the login/logout functionality provided in roundup.cgi_client.Client -(classic schema) or roundup.cgi_client.ExtendedClient (extended schema) and -modify your instance code apropriately. - - -0.3.x Password encoding ------------------------ -This release also introduces encoding of passwords in the database. If you -have not edited the dbinit.py file in your instance home directory, you may -simply copy the new dbinit.py file from the core code. If you used the -classic schema, the interfaces file is in:: - - <roundup source>/roundup/templates/classic/dbinit.py - -If you used the extended schema, the file is in:: - - <roundup source>/roundup/templates/extended/dbinit.py - - -If you have modified your dbinit.py file, you may use encoded passwords: - -1. Edit the dbinit.py file in your instance home directory - a. At the first code line of the open() function:: - - from roundup.hyperdb import String, Date, Link, Multilink - - alter to include Password, as so:: - - from roundup.hyperdb import String, Password, Date, Link, Multilink - - b. Where the password property is defined (around line 66):: - - user = Class(db, "user", - username=String(), password=String(), - address=String(), realname=String(), - phone=String(), organisation=String()) - user.setkey("username") - - alter the "password=String()" to "password=Password()":: - - user = Class(db, "user", - username=String(), password=Password(), - address=String(), realname=String(), - phone=String(), organisation=String()) - user.setkey("username") - -2. Any existing passwords in the database will remain cleartext until they - are edited. It is recommended that at a minimum the admin password be - changed immediately:: - - roundup-admin -i <instance home> set user1 password=<new password> - - -0.3.x Configuration -------------------- -FILTER_POSITION, ANONYMOUS_ACCESS, ANONYMOUS_REGISTER have been added to -the instance_config.py. Simplest solution is to copy the default values from -template in the core source. - -MESSAGES_TO_AUTHOR has been added to the IssueClass in dbinit.py. Set to 'yes' -to send nosy messages to the author. Default behaviour is to not send nosy -messages to the author. You will need to add MESSAGES_TO_AUTHOR to your -dbinit.py in your instance home. - - -0.3.x CGI script roundup.cgi ----------------------------- -There have been some structural changes to the roundup.cgi script - you will -need to install it again from the cgi-bin directory of the source -distribution. Make sure you update the ROUNDUP_INSTANCE_HOMES after the -copy. - - -.. _`customisation documentation`: customizing.html -.. _`security documentation`: security.html
--- a/doc/user_guide.txt Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,374 +0,0 @@ -========== -User Guide -========== - -:Version: $Revision: 1.9 $ - -.. contents:: - -Note: this document will refer to *issues* as the primary store of information -in the tracker. This is the default of the classic template, bubt may vary in -any given installation. - -Your Tracker in a Nutshell -========================== - -Your tracker holds information about issues in bundles we call *items*. An -item may be an *issue* (a bug or feature request) or a *user*. The issue-ness or -user-ness is called the item's *class*. So, for bug reports and features, the -class is "issue", and for users the class is "user". - -Each item in the tracker has an id number that identifies it along with its -item class. To identify a particular issue or user, we combine the class with -the number to create a unique label, so that user 1 (who, incidentally, is -*always* the "admin" user) is referred to as "user1". Issue number 315 is -referred to as "issue315". We call that label the item's *designator*. - -Accessing the Tracker ---------------------- - -You may access your tracker through one of three ways: - -1. through the `web interface`_, -2. through the `e-mail gateway`_, or -3. using the `command line tool`_. - -The last is usually only used by administrators. Most users will use the web -and email interfaces. All three are explained below. - - -Web Interface -============= - -Note: this document contains screenshots of the default look and feel. Your -site may have a slightly (or very) different look, but the functionality will -be very similar, and the concepts still hold. - -The web interface is broken up into the following parts: - -1. `lists of items`_, -2. `display, edit or entry of an item`_, and -3. `searching page`_. - - -Lists of Items --------------- - -The first thing you'll see when you log into Roundup will be a list of open -(ie. not resolved) issues. This list has been generated by a bunch of controls -`under the covers`_ but for now, you can see something like: - -.. img: images/index_logged_out.png - -The screen is divided up into three sections: - -.. img: images/page_layout.png - -you may either register or log in. Registration takes you to: - -.. img: images/registration.png - -Once you're logged in, the screen changes slightly to: - -.. img: images/index_logged_in.png - -Note that the sidebar menu has changed slightly, so you can now get to your -"My Details" page: - -.. img: images/my_details.png - -Note the new information on this page - the history. - - -Display, edit or entry of an item ---------------------------------- - -Create a new issue with "create new" under the issue subheading. This will -take you to: - -.. img: images/new_issue.png - -The `nosy list`_ is explained below. -Enter some information and click "submit new entry" and you'll be rewarded -with: - -.. img: images/new_issue_created.png - -or, if you don't enter all the required information (or some other error -occurs) you'll get something like: - -.. img: images/new_issue_error.png - - -Searching Page --------------- - -XXX: some information about how searching works - - -Under the covers ----------------- - -Index views may be modified by the following arguments: - -========== ============================================================= -Argument Description -========== ============================================================= -:sort sort by prop name, optionally preceeded with '-' - to give descending or nothing for ascending sorting. -:group group by prop name, optionally preceeded with '-' or - to sort in descending or nothing for ascending order. -:filter selects which props should be displayed in the filter - section. Default is all. -:columns selects the columns that should be displayed. - Default is all. -propname selects the values the item properties given by propname - must have (very basic search/filter). -========== ============================================================= - -Access Controls ---------------- - -XXX - - -E-Mail Gateway -============== - -E-mail sent to Roundup is examined for several pieces of information: - -1. `subject-line information`_ identifying the purpose of the e-mail -2. `e-mail message content`_ which is to be extracted -3. e-mail attachments which should be associated with the message - -Subject-line information ------------------------- - -The subject line of the incoming message is examined to find one of: - -1. the item that the message is responding to, -2. the type of item the message should create, or -3. we default the item class and try some trickiness - -If the subject line contains a prefix in ``[square brackets]`` then we're -looking at case 1 or 2 above. Note that any "re:" or "fwd:" prefixes are -stripped off the subject line before we start looking for real information. - -If an item designator (class name and id number, for example ``issue123``) -is found there, a new "msg" item is added to the "messages" property for -that item, and any new "file" items are added to the "files" property for -the item. - -If just an item class name is found there, we attempt to create a new item of -that class with its "messages" property initialized to contain the new "msg" -item and its "files" property initialized to contain any new "file" items. - -The third case above - where no ``[information]`` is provided, the tracker's -``MAIL_DEFAULT_CLASS`` configuration variable defines what class of item -the message relates to. We try to match the subject line to an existing -item of the default class, and if there's a match, the message is related to -that matched item. If not, then a new item of the default class is created. - -Setting Properties -~~~~~~~~~~~~~~~~~~ - -The e-mail interface also provides a simple way to set properties on items. At -the end of the subject line, propname=value pairs can be specified in square -brackets, using the same conventions as for the roundup set shell command. - -For example, - -- setting the priority of an issue:: - - Subject: Re: [issue1] the coffee machine is broken! [priority=urgent] - -- adding yourself to a nosy list:: - - Subject: Re: [issue2] we're out of widgets [nosy=+richard] - -- setting the nosy list to just you:: - - Subject: Re: [issue2] we're out of widgets [nosy=richard] - -- removing yourself from a nosy list:: - - Subject: Re: [issue2] we're out of widgets [nosy=-richard] - -In all cases, the message relates to issue 2. The ``Re:`` prefix is stripped -off. - - -E-Mail Message Content ----------------------- - -Roundup only associates plain text (MIME type ``text/plain``) as messages for -items. Any other parts of a message are associated as downloadable files. If -no plain text part is found, the message is rejected. - -To do this, incoming messages are examined for multiple parts: - -* In a multipart/mixed message or part, each subpart is extracted and - examined. The text/plain subparts are assembled to form the textual body - of the message, to be stored in the file associated with a "msg" class - item. Any parts of other types are each stored in separate files and - given "file" class items that are linked to the "msg" item. -* In a multipart/alternative message or part, we look for a text/plain - subpart and ignore the other parts. - -If the message is a response to a previous message, and contains quoted -sections, then these will be stripped out of the message if the -``EMAIL_KEEP_QUOTED_TEXT`` configuration variable is set to ``'no'``. - -Message summary -~~~~~~~~~~~~~~~ - -The "summary" property on message items is taken from the first non-quoting -section in the message body. The message body is divided into sections by blank -lines. Sections where the second and all subsequent lines begin with a ">" or -"|" character are considered "quoting sections". The first line of the first -non-quoting section becomes the summary of the message. - - -Address handling ----------------- - -All of the addresses in the ``To:`` and ``Cc:`` headers of the incoming -message are -looked up among the tracker users, and the corresponding users are placed -in the -"recipients" property on the new "msg" item. The address in the ``From:`` header -similarly determines the "author" property of the new "msg" item. The default -handling for addresses that don't have corresponding users is to create new -users with no passwords and a username equal to the address. - -The addresses mentioned in the ``To:``, ``From:`` and ``Cc:`` headers of -the message may be added to the `nosy list`_ depending on: - -``ADD_AUTHOR_TO_NOSY`` - Does the author of a message get placed on the nosy list automatically? - If 'new' is used, then the author will only be added when a message - creates a new issue. If 'yes', then the author will be added on followups - too. If 'no', they're never added to the nosy. - -``ADD_RECIPIENTS_TO_NOSY`` - Do the recipients (To:, Cc:) of a message get placed on the nosy list? - If 'new' is used, then the recipients will only be added when a message - creates a new issue. If 'yes', then the recipients will be added on - followups too. If 'no', they're never added to the nosy. - - -Nosy List -~~~~~~~~~ - -Roundup watches for additions to the "messages" property of items. - -When a new message is added, it is sent to all the users -on the "nosy" list for the item that are not already on the "recipients" list -of the message. Those users are then appended to the "recipients" property on -the message, so multiple copies of a message are never sent to the same user. -The journal recorded by the hyperdatabase on the "recipients" property then -provides a log of when the message was sent to whom. - -If the author of the message is also in the nosy list for the item that the -message is attached to, then the config var ``MESSAGES_TO_AUTHOR`` is queried -to determine if they get a nosy list copy of the message too. - - -Command Line Tool -================= - -The basic usage is:: - - Help: - roundup-admin -h - roundup-admin help -- this help - roundup-admin help <command> -- command-specific help - roundup-admin help all -- all available help - - Options: - -i instance home -- specify the issue tracker "home directory" to administer - -u -- the user[:password] to use for commands - -c -- when outputting lists of data, just comma-separate them - - Commands: - commit - create classname property=value ... - display designator - export [class[,class]] export_dir - find classname propname=value ... - get property designator[,designator]* - help topic - history designator - import import_dir - initialise [adminpw] - install [template [backend [admin password]]] - list classname [property] - pack period | date - reindex - retire designator[,designator]* - rollback - security [Role name] - set designator[,designator]* propname=value ... - specification classname - table classname [property[,property]*] - -Commands may be abbreviated as long as the abbreviation matches only one -command, e.g. l == li == lis == list. - -All commands (except help) require a tracker specifier. This is just the -path to the roundup tracker you're working with. A roundup tracker is where -roundup keeps the database and configuration file that defines an issue -tracker. It may be thought of as the issue tracker's "home directory". -It may be specified in the environment variable ``TRACKER_HOME`` or on -the command line as "``-i tracker``". - -A designator is a classname and an itemid concatenated, eg. bug1, user10, ... -Property values are represented as strings in command arguments and in the printed -results: - -- Strings are, well, strings. -- Password values will display as their encoded value. -- Date values are printed in the full date format in the local time zone, - and accepted in the full format or any of the partial formats explained - below.:: - - Input of... Means... - "2000-04-17.03:45" 2000-04-17.08:45:00 - "2000-04-17" 2000-04-17.00:00:00 - "01-25" yyyy-01-25.00:00:00 - "08-13.22:13" yyyy-08-14.03:13:00 - "11-07.09:32:43" yyyy-11-07.14:32:43 - "14:25" yyyy-mm-dd.19:25:00 - "8:47:11" yyyy-mm-dd.13:47:11 - "." "right now" - -- Link values are printed as item designators. When given as an argument, - item designators and key strings are both accepted. -- Multilink values are printed as lists of item designators joined by - commas. When given as an argument, item designators and key strings are - both accepted; an empty string, a single item, or a list of items joined - by commas is accepted. - -When multiple items are specified to the roundup get or roundup set -commands, the specified properties are retrieved or set on all the listed -items. When multiple results are returned by the roundup get or roundup -find commands, they are printed one per line (default) or joined by commas -(with the "``-c``" option). - -Where the command changes data, a login name/password is required. The login may -be specified as either "``name``" or "``name:password``". - -- ``ROUNDUP_LOGIN`` environment variable -- the "``-u``" command-line option - -If either the name or password is not supplied, they are obtained from the -command-line. - - - ------------------ - -Back to `Table of Contents`_ - -.. _`Table of Contents`: index.html -
--- a/frontends/ZRoundup/ZRoundup.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,209 +0,0 @@ -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: ZRoundup.py,v 1.13 2002-09-10 03:01:18 richard Exp $ -# -''' ZRoundup module - exposes the roundup web interface to Zope - -This frontend works by providing a thin layer that sits between Zope and the -regular CGI interface of roundup, providing the web frontend with the minimum -of effort. - -This means that the regular CGI interface does all authentication quite -independently of Zope. The roundup code is kept in memory though, and it -runs in the same server as all your other Zope stuff, so it does have _some_ -advantages over regular CGI :) - -It also means that any requests which specify :filter, :columns or :sort -_must_ be done using a GET, so that this interface can re-parse the -QUERY_STRING. Zope interprets the ':' as a special character, and the special -args are lost to it. -''' - -import urlparse - -from Globals import InitializeClass, HTMLFile -from OFS.SimpleItem import Item -from OFS.PropertyManager import PropertyManager -from Acquisition import Implicit -from Persistence import Persistent -from AccessControl import ClassSecurityInfo -from AccessControl import ModuleSecurityInfo -modulesecurity = ModuleSecurityInfo() - -import roundup.instance -from roundup.cgi.client import NotFound - -modulesecurity.declareProtected('View management screens', - 'manage_addZRoundupForm') -manage_addZRoundupForm = HTMLFile('dtml/manage_addZRoundupForm', globals()) - -modulesecurity.declareProtected('Add Z Roundups', 'manage_addZRoundup') -def manage_addZRoundup(self, id, instance_home, REQUEST): - """Add a ZRoundup product """ - # validate the instance_home - roundup.instance.open(instance_home) - self._setObject(id, ZRoundup(id, instance_home)) - return self.manage_main(self, REQUEST) - -class RequestWrapper: - '''Make the Zope RESPONSE look like a BaseHTTPServer - ''' - def __init__(self, RESPONSE): - self.RESPONSE = RESPONSE - self.wfile = self.RESPONSE - def send_response(self, status): - self.RESPONSE.setStatus(status) - def send_header(self, header, value): - self.RESPONSE.addHeader(header, value) - def end_headers(self): - # not needed - the RESPONSE object handles this internally on write() - pass - -class FormItem: - '''Make a Zope form item look like a cgi.py one - ''' - def __init__(self, value): - self.value = value - if hasattr(self.value, 'filename'): - self.filename = self.value.filename - self.file = self.value - -class FormWrapper: - '''Make a Zope form dict look like a cgi.py one - ''' - def __init__(self, form): - self.form = form - def __getitem__(self, item): - return FormItem(self.form[item]) - def has_key(self, item): - return self.form.has_key(item) - def keys(self): - return self.form.keys() - -class ZRoundup(Item, PropertyManager, Implicit, Persistent): - '''An instance of this class provides an interface between Zope and - roundup for one roundup instance - ''' - meta_type = 'Z Roundup' - security = ClassSecurityInfo() - - def __init__(self, id, instance_home): - self.id = id - self.instance_home = instance_home - - # define the properties that define this object - _properties = ( - {'id':'id', 'type': 'string', 'mode': 'w'}, - {'id':'instance_home', 'type': 'string', 'mode': 'w'}, - ) - property_extensible_schema__ = 0 - - # define the tabs for the management interface - manage_options= PropertyManager.manage_options + ( - {'label': 'View', 'action':'index_html'}, - ) + Item.manage_options - - icon = "misc_/ZRoundup/icon" - - security.declarePrivate('_opendb') - def _opendb(self): - '''Open the roundup instance database for a transaction. - ''' - instance = roundup.instance.open(self.instance_home) - request = RequestWrapper(self.REQUEST['RESPONSE']) - env = self.REQUEST.environ - - # figure out the path components to set - url = urlparse.urlparse( self.absolute_url() ) - path = url[2] - path_components = path.split( '/' ) - - # special case when roundup is '/' in this virtual host, - if path == "/" : - env['SCRIPT_NAME'] = "/" - env['TRACKER_NAME'] = '' - else : - # all but the last element is the path - env['SCRIPT_NAME'] = '/'.join( path_components[:-1] ) - # the last element is the name - env['TRACKER_NAME'] = path_components[-1] - - if env['REQUEST_METHOD'] == 'GET': - # force roundup to re-parse the request because Zope fiddles - # with it and we lose all the :filter, :columns, etc goodness - form = None - else: - # For some reason, CRs are embeded in multiline notes. - # It doesn't occur with apache/roundup.cgi, though. - form = self.REQUEST.form - if form.has_key( '__note' ) : - form['__note'] = form['__note'].replace( '\r' , '' ) - form = FormWrapper(form) - - return instance.Client(instance, request, env, form) - - - security.declareProtected('View', 'index_html') - def index_html(self): - '''Alias index_html to roundup's index - ''' - - # Redirect misdirected requests -- bugs 558867 , 565992 - - # PATH_INFO, as defined by the CGI spec, has the *real* request path - orig_path = self.REQUEST.environ[ 'PATH_INFO' ] - if orig_path[-1] != '/' : - url = urlparse.urlparse( self.absolute_url() ) - url = list( url ) # make mutable - url[2] = url[2]+'/' # patch - url = urlparse.urlunparse( url ) # reassemble - RESPONSE = self.REQUEST.RESPONSE - RESPONSE.setStatus( "MovedPermanently" ) # 301 - RESPONSE.setHeader( "Location" , url ) - return RESPONSE - - client = self._opendb() - # fake the path that roundup should use - client.split_path = ['index'] - return client.main() - - def __getitem__(self, item): - '''All other URL accesses are passed throuh to roundup - ''' - try: - client = self._opendb() - # fake the path that roundup should use - client.split_path = [item] - # and call roundup to do something - client.main() - return '' - except NotFound: - raise 'NotFound', self.REQUEST.URL - pass - except: - import traceback - traceback.print_exc() - # all other exceptions in roundup are valid - raise - raise KeyError, item - - -InitializeClass(ZRoundup) -modulesecurity.apply(globals()) - - -# vim: set filetype=python ts=4 sw=4 et si
--- a/frontends/ZRoundup/__init__.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,56 +0,0 @@ -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: __init__.py,v 1.3 2002-09-10 03:01:18 richard Exp $ -# -__version__='1.0' - -import os -# figure where ZRoundup is installed -here = None -if os.environ.has_key('TRACKER_HOME'): - here = os.environ['TRACKER_HOME'] - path = os.path.join(here, 'Products', 'ZRoundup') - if not os.path.exists(path): - path = os.path.join(here, 'lib', 'python', 'Products', 'ZRoundup') - if not os.path.exists(path): - here = None -if here is None: - from __main__ import here - path = os.path.join(here, 'Products', 'ZRoundup') - if not os.path.exists(path): - path = os.path.join(here, 'lib', 'python', 'Products', 'ZRoundup') - if not os.path.exists(path): - raise ValueError, "Can't determine where ZRoundup is installed" - -# product initialisation -import ZRoundup -def initialize(context): - context.registerClass( - ZRoundup, meta_type = 'Z Roundup', - constructors = ( - ZRoundup.manage_addZRoundupForm, ZRoundup.manage_addZRoundup - ) - ) - -# set up the icon -from ImageFile import ImageFile -misc_ = { - 'icon': ImageFile('icons/tick_symbol.gif', path), -} - - -# vim: set filetype=python ts=4 sw=4 et si
--- a/roundup/__init__.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,70 +0,0 @@ -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: __init__.py,v 1.13 2002-09-26 04:19:53 richard Exp $ - -__doc__ = ''' -This is a simple-to-use and -install issue-tracking system with -command-line, web and e-mail interfaces. - -Roundup manages a number of issues (with properties such as -"description", "priority", and so on) and provides the ability to (a) submit -new issues, (b) find and edit existing issues, and (c) discuss issues with -other participants. The system will facilitate communication among the -participants by managing discussions and notifying interested parties when -issues are edited. - -Roundup's structure is that of a cake: - - _________________________________________________________________________ -| E-mail Client | Web Browser | Detector Scripts | Shell | -|------------------+-----------------+----------------------+-------------| -| E-mail User | Web User | Detector | Command | -|-------------------------------------------------------------------------| -| Roundup Database Layer | -|-------------------------------------------------------------------------| -| Hyperdatabase Layer | -|-------------------------------------------------------------------------| -| Storage Layer | - ------------------------------------------------------------------------- - -The first layer represents the users (chocolate). -The second layer is the Roundup interface to the users (vanilla). -The third and fourth layers are the internal Roundup database storage - mechanisms (strawberry). -The final, lowest layer is the underlying database storage (rum). - -These are implemented in the code in the following manner: - E-mail User: roundup-mailgw and roundup.mailgw - Web User: cgi-bin/roundup.cgi or roundup-server over - roundup.cgi_client, roundup.cgitb and roundup.htmltemplate - Detector: roundup.roundupdb and templates/<template>/detectors - Command: roundup-admin - Roundup DB: roundup.roundupdb - Hyper DB: roundup.hyperdb, roundup.date - Storage: roundup.backends.* - -Additionally, there is a directory of unit tests in "test". - -For more information, see the original overview and specification documents -written by Ka-Ping Yee in the "doc" directory. If nothing else, it has a -much prettier cake :) -''' - -__version__ = '0.5.0pr1' - -# vim: set filetype=python ts=4 sw=4 et si
--- a/roundup/admin.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1190 +0,0 @@ -#! /usr/bin/env python -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: admin.py,v 1.34 2002-09-26 07:41:54 richard Exp $ - -import sys, os, getpass, getopt, re, UserDict, shlex, shutil -try: - import csv -except ImportError: - csv = None -from roundup import date, hyperdb, roundupdb, init, password, token -from roundup import __version__ as roundup_version -import roundup.instance -from roundup.i18n import _ - -class CommandDict(UserDict.UserDict): - '''Simple dictionary that lets us do lookups using partial keys. - - Original code submitted by Engelbert Gruber. - ''' - _marker = [] - def get(self, key, default=_marker): - if self.data.has_key(key): - return [(key, self.data[key])] - keylist = self.data.keys() - keylist.sort() - l = [] - for ki in keylist: - if ki.startswith(key): - l.append((ki, self.data[ki])) - if not l and default is self._marker: - raise KeyError, key - return l - -class UsageError(ValueError): - pass - -class AdminTool: - - def __init__(self): - self.commands = CommandDict() - for k in AdminTool.__dict__.keys(): - if k[:3] == 'do_': - self.commands[k[3:]] = getattr(self, k) - self.help = {} - for k in AdminTool.__dict__.keys(): - if k[:5] == 'help_': - self.help[k[5:]] = getattr(self, k) - self.tracker_home = '' - self.db = None - - def get_class(self, classname): - '''Get the class - raise an exception if it doesn't exist. - ''' - try: - return self.db.getclass(classname) - except KeyError: - raise UsageError, _('no such class "%(classname)s"')%locals() - - def props_from_args(self, args): - props = {} - for arg in args: - if arg.find('=') == -1: - raise UsageError, _('argument "%(arg)s" not propname=value')%locals() - try: - key, value = arg.split('=') - except ValueError: - raise UsageError, _('argument "%(arg)s" not propname=value')%locals() - if value: - props[key] = value - else: - props[key] = None - return props - - def usage(self, message=''): - if message: - message = _('Problem: %(message)s)\n\n')%locals() - print _('''%(message)sUsage: roundup-admin [options] <command> <arguments> - -Options: - -i instance home -- specify the issue tracker "home directory" to administer - -u -- the user[:password] to use for commands - -c -- when outputting lists of data, just comma-separate them - -Help: - roundup-admin -h - roundup-admin help -- this help - roundup-admin help <command> -- command-specific help - roundup-admin help all -- all available help -''')%locals() - self.help_commands() - - def help_commands(self): - print _('Commands:'), - commands = [''] - for command in self.commands.values(): - h = command.__doc__.split('\n')[0] - commands.append(' '+h[7:]) - commands.sort() - commands.append(_('Commands may be abbreviated as long as the abbreviation matches only one')) - commands.append(_('command, e.g. l == li == lis == list.')) - print '\n'.join(commands) - print - - def help_commands_html(self, indent_re=re.compile(r'^(\s+)\S+')): - commands = self.commands.values() - def sortfun(a, b): - return cmp(a.__name__, b.__name__) - commands.sort(sortfun) - for command in commands: - h = command.__doc__.split('\n') - name = command.__name__[3:] - usage = h[0] - print _(''' -<tr><td valign=top><strong>%(name)s</strong></td> - <td><tt>%(usage)s</tt><p> -<pre>''')%locals() - indent = indent_re.match(h[3]) - if indent: indent = len(indent.group(1)) - for line in h[3:]: - if indent: - print line[indent:] - else: - print line - print _('</pre></td></tr>\n') - - def help_all(self): - print _(''' -All commands (except help) require a tracker specifier. This is just the path -to the roundup tracker you're working with. A roundup tracker is where -roundup keeps the database and configuration file that defines an issue -tracker. It may be thought of as the issue tracker's "home directory". It may -be specified in the environment variable TRACKER_HOME or on the command -line as "-i tracker". - -A designator is a classname and a nodeid concatenated, eg. bug1, user10, ... - -Property values are represented as strings in command arguments and in the -printed results: - . Strings are, well, strings. - . Date values are printed in the full date format in the local time zone, and - accepted in the full format or any of the partial formats explained below. - . Link values are printed as node designators. When given as an argument, - node designators and key strings are both accepted. - . Multilink values are printed as lists of node designators joined by commas. - When given as an argument, node designators and key strings are both - accepted; an empty string, a single node, or a list of nodes joined by - commas is accepted. - -When property values must contain spaces, just surround the value with -quotes, either ' or ". A single space may also be backslash-quoted. If a -valuu must contain a quote character, it must be backslash-quoted or inside -quotes. Examples: - hello world (2 tokens: hello, world) - "hello world" (1 token: hello world) - "Roch'e" Compaan (2 tokens: Roch'e Compaan) - Roch\'e Compaan (2 tokens: Roch'e Compaan) - address="1 2 3" (1 token: address=1 2 3) - \\ (1 token: \) - \n\r\t (1 token: a newline, carriage-return and tab) - -When multiple nodes are specified to the roundup get or roundup set -commands, the specified properties are retrieved or set on all the listed -nodes. - -When multiple results are returned by the roundup get or roundup find -commands, they are printed one per line (default) or joined by commas (with -the -c) option. - -Where the command changes data, a login name/password is required. The -login may be specified as either "name" or "name:password". - . ROUNDUP_LOGIN environment variable - . the -u command-line option -If either the name or password is not supplied, they are obtained from the -command-line. - -Date format examples: - "2000-04-17.03:45" means <Date 2000-04-17.08:45:00> - "2000-04-17" means <Date 2000-04-17.00:00:00> - "01-25" means <Date yyyy-01-25.00:00:00> - "08-13.22:13" means <Date yyyy-08-14.03:13:00> - "11-07.09:32:43" means <Date yyyy-11-07.14:32:43> - "14:25" means <Date yyyy-mm-dd.19:25:00> - "8:47:11" means <Date yyyy-mm-dd.13:47:11> - "." means "right now" - -Command help: -''') - for name, command in self.commands.items(): - print _('%s:')%name - print _(' '), command.__doc__ - - def do_help(self, args, nl_re=re.compile('[\r\n]'), - indent_re=re.compile(r'^(\s+)\S+')): - '''Usage: help topic - Give help about topic. - - commands -- list commands - <command> -- help specific to a command - initopts -- init command options - all -- all available help - ''' - if len(args)>0: - topic = args[0] - else: - topic = 'help' - - - # try help_ methods - if self.help.has_key(topic): - self.help[topic]() - return 0 - - # try command docstrings - try: - l = self.commands.get(topic) - except KeyError: - print _('Sorry, no help for "%(topic)s"')%locals() - return 1 - - # display the help for each match, removing the docsring indent - for name, help in l: - lines = nl_re.split(help.__doc__) - print lines[0] - indent = indent_re.match(lines[1]) - if indent: indent = len(indent.group(1)) - for line in lines[1:]: - if indent: - print line[indent:] - else: - print line - return 0 - - def help_initopts(self): - import roundup.templates - templates = roundup.templates.listTemplates() - print _('Templates:'), ', '.join(templates) - import roundup.backends - backends = roundup.backends.__all__ - print _('Back ends:'), ', '.join(backends) - - def do_install(self, tracker_home, args): - '''Usage: install [template [backend [admin password]]] - Install a new Roundup tracker. - - The command will prompt for the tracker home directory (if not supplied - through TRACKER_HOME or the -i option). The template, backend and admin - password may be specified on the command-line as arguments, in that - order. - - The initialise command must be called after this command in order - to initialise the tracker's database. You may edit the tracker's - initial database contents before running that command by editing - the tracker's dbinit.py module init() function. - - See also initopts help. - ''' - if len(args) < 1: - raise UsageError, _('Not enough arguments supplied') - - # make sure the tracker home can be created - parent = os.path.split(tracker_home)[0] - if not os.path.exists(parent): - raise UsageError, _('Instance home parent directory "%(parent)s"' - ' does not exist')%locals() - - # select template - import roundup.templates - templates = roundup.templates.listTemplates() - template = len(args) > 1 and args[1] or '' - if template not in templates: - print _('Templates:'), ', '.join(templates) - while template not in templates: - template = raw_input(_('Select template [classic]: ')).strip() - if not template: - template = 'classic' - - # select hyperdb backend - import roundup.backends - backends = roundup.backends.__all__ - backend = len(args) > 2 and args[2] or '' - if backend not in backends: - print _('Back ends:'), ', '.join(backends) - while backend not in backends: - backend = raw_input(_('Select backend [anydbm]: ')).strip() - if not backend: - backend = 'anydbm' - # XXX perform a unit test based on the user's selections - - # install! - init.install(tracker_home, template, backend) - - print _(''' - You should now edit the tracker configuration file: - %(config_file)s - ... at a minimum, you must set MAILHOST, MAIL_DOMAIN and ADMIN_EMAIL. - - If you wish to modify the default schema, you should also edit the database - initialisation file: - %(database_config_file)s - ... see the documentation on customizing for more information. -''')%{ - 'config_file': os.path.join(tracker_home, 'config.py'), - 'database_config_file': os.path.join(tracker_home, 'dbinit.py') -} - return 0 - - - def do_initialise(self, tracker_home, args): - '''Usage: initialise [adminpw] - Initialise a new Roundup tracker. - - The administrator details will be set at this step. - - Execute the tracker's initialisation function dbinit.init() - ''' - # password - if len(args) > 1: - adminpw = args[1] - else: - adminpw = '' - confirm = 'x' - while adminpw != confirm: - adminpw = getpass.getpass(_('Admin Password: ')) - confirm = getpass.getpass(_(' Confirm: ')) - - # make sure the tracker home is installed - if not os.path.exists(tracker_home): - raise UsageError, _('Instance home does not exist')%locals() - if not os.path.exists(os.path.join(tracker_home, 'html')): - raise UsageError, _('Instance has not been installed')%locals() - - # is there already a database? - if os.path.exists(os.path.join(tracker_home, 'db')): - print _('WARNING: The database is already initialised!') - print _('If you re-initialise it, you will lose all the data!') - ok = raw_input(_('Erase it? Y/[N]: ')).strip() - if ok.lower() != 'y': - return 0 - - # nuke it - shutil.rmtree(os.path.join(tracker_home, 'db')) - - # GO - init.initialise(tracker_home, adminpw) - - return 0 - - - def do_get(self, args): - '''Usage: get property designator[,designator]* - Get the given property of one or more designator(s). - - Retrieves the property value of the nodes specified by the designators. - ''' - if len(args) < 2: - raise UsageError, _('Not enough arguments supplied') - propname = args[0] - designators = args[1].split(',') - l = [] - for designator in designators: - # decode the node designator - try: - classname, nodeid = hyperdb.splitDesignator(designator) - except hyperdb.DesignatorError, message: - raise UsageError, message - - # get the class - cl = self.get_class(classname) - try: - if self.comma_sep: - l.append(cl.get(nodeid, propname)) - else: - print cl.get(nodeid, propname) - except IndexError: - raise UsageError, _('no such %(classname)s node "%(nodeid)s"')%locals() - except KeyError: - raise UsageError, _('no such %(classname)s property ' - '"%(propname)s"')%locals() - if self.comma_sep: - print ','.join(l) - return 0 - - - def do_set(self, args, pwre = re.compile(r'{(\w+)}(.+)')): - '''Usage: set [items] property=value property=value ... - Set the given properties of one or more items(s). - - The items may be specified as a class or as a comma-separeted - list of item designators (ie "designator[,designator,...]"). - - This command sets the properties to the values for all designators - given. If the value is missing (ie. "property=") then the property is - un-set. - ''' - if len(args) < 2: - raise UsageError, _('Not enough arguments supplied') - from roundup import hyperdb - - designators = args[0].split(',') - if len(designators) == 1: - designator = designators[0] - try: - designator = hyperdb.splitDesignator(designator) - designators = [designator] - except hyperdb.DesignatorError: - cl = self.get_class(designator) - designators = [(designator, x) for x in cl.list()] - else: - try: - designators = [hyperdb.splitDesignator(x) for x in designators] - except hyperdb.DesignatorError, message: - raise UsageError, message - - # get the props from the args - props = self.props_from_args(args[1:]) - - # now do the set for all the nodes - for classname, itemid in designators: - cl = self.get_class(classname) - - properties = cl.getprops() - for key, value in props.items(): - proptype = properties[key] - if isinstance(proptype, hyperdb.Multilink): - if value is None: - props[key] = [] - else: - props[key] = value.split(',') - elif value is None: - continue - elif isinstance(proptype, hyperdb.String): - continue - elif isinstance(proptype, hyperdb.Password): - m = pwre.match(value) - if m: - # password is being given to us encrypted - p = password.Password() - p.scheme = m.group(1) - p.password = m.group(2) - props[key] = p - else: - props[key] = password.Password(value) - elif isinstance(proptype, hyperdb.Date): - try: - props[key] = date.Date(value) - except ValueError, message: - raise UsageError, '"%s": %s'%(value, message) - elif isinstance(proptype, hyperdb.Interval): - try: - props[key] = date.Interval(value) - except ValueError, message: - raise UsageError, '"%s": %s'%(value, message) - elif isinstance(proptype, hyperdb.Link): - props[key] = value - elif isinstance(proptype, hyperdb.Boolean): - props[key] = value.lower() in ('yes', 'true', 'on', '1') - elif isinstance(proptype, hyperdb.Number): - props[key] = int(value) - - # try the set - try: - apply(cl.set, (itemid, ), props) - except (TypeError, IndexError, ValueError), message: - import traceback; traceback.print_exc() - raise UsageError, message - return 0 - - def do_find(self, args): - '''Usage: find classname propname=value ... - Find the nodes of the given class with a given link property value. - - Find the nodes of the given class with a given link property value. The - value may be either the nodeid of the linked node, or its key value. - ''' - if len(args) < 1: - raise UsageError, _('Not enough arguments supplied') - classname = args[0] - # get the class - cl = self.get_class(classname) - - # handle the propname=value argument - props = self.props_from_args(args[1:]) - - # if the value isn't a number, look up the linked class to get the - # number - for propname, value in props.items(): - num_re = re.compile('^\d+$') - if not num_re.match(value): - # get the property - try: - property = cl.properties[propname] - except KeyError: - raise UsageError, _('%(classname)s has no property ' - '"%(propname)s"')%locals() - - # make sure it's a link - if (not isinstance(property, hyperdb.Link) and not - isinstance(property, hyperdb.Multilink)): - raise UsageError, _('You may only "find" link properties') - - # get the linked-to class and look up the key property - link_class = self.db.getclass(property.classname) - try: - props[propname] = link_class.lookup(value) - except TypeError: - raise UsageError, _('%(classname)s has no key property"')%{ - 'classname': link_class.classname} - - # now do the find - try: - if self.comma_sep: - print ','.join(apply(cl.find, (), props)) - else: - print apply(cl.find, (), props) - except KeyError: - raise UsageError, _('%(classname)s has no property ' - '"%(propname)s"')%locals() - except (ValueError, TypeError), message: - raise UsageError, message - return 0 - - def do_specification(self, args): - '''Usage: specification classname - Show the properties for a classname. - - This lists the properties for a given class. - ''' - if len(args) < 1: - raise UsageError, _('Not enough arguments supplied') - classname = args[0] - # get the class - cl = self.get_class(classname) - - # get the key property - keyprop = cl.getkey() - for key, value in cl.properties.items(): - if keyprop == key: - print _('%(key)s: %(value)s (key property)')%locals() - else: - print _('%(key)s: %(value)s')%locals() - - def do_display(self, args): - '''Usage: display designator - Show the property values for the given node. - - This lists the properties and their associated values for the given - node. - ''' - if len(args) < 1: - raise UsageError, _('Not enough arguments supplied') - - # decode the node designator - try: - classname, nodeid = hyperdb.splitDesignator(args[0]) - except hyperdb.DesignatorError, message: - raise UsageError, message - - # get the class - cl = self.get_class(classname) - - # display the values - for key in cl.properties.keys(): - value = cl.get(nodeid, key) - print _('%(key)s: %(value)s')%locals() - - def do_create(self, args, pwre = re.compile(r'{(\w+)}(.+)')): - '''Usage: create classname property=value ... - Create a new entry of a given class. - - This creates a new entry of the given class using the property - name=value arguments provided on the command line after the "create" - command. - ''' - if len(args) < 1: - raise UsageError, _('Not enough arguments supplied') - from roundup import hyperdb - - classname = args[0] - - # get the class - cl = self.get_class(classname) - - # now do a create - props = {} - properties = cl.getprops(protected = 0) - if len(args) == 1: - # ask for the properties - for key, value in properties.items(): - if key == 'id': continue - name = value.__class__.__name__ - if isinstance(value , hyperdb.Password): - again = None - while value != again: - value = getpass.getpass(_('%(propname)s (Password): ')%{ - 'propname': key.capitalize()}) - again = getpass.getpass(_(' %(propname)s (Again): ')%{ - 'propname': key.capitalize()}) - if value != again: print _('Sorry, try again...') - if value: - props[key] = value - else: - value = raw_input(_('%(propname)s (%(proptype)s): ')%{ - 'propname': key.capitalize(), 'proptype': name}) - if value: - props[key] = value - else: - props = self.props_from_args(args[1:]) - - # convert types - for propname, value in props.items(): - # get the property - try: - proptype = properties[propname] - except KeyError: - raise UsageError, _('%(classname)s has no property ' - '"%(propname)s"')%locals() - - if isinstance(proptype, hyperdb.Date): - try: - props[propname] = date.Date(value) - except ValueError, message: - raise UsageError, _('"%(value)s": %(message)s')%locals() - elif isinstance(proptype, hyperdb.Interval): - try: - props[propname] = date.Interval(value) - except ValueError, message: - raise UsageError, _('"%(value)s": %(message)s')%locals() - elif isinstance(proptype, hyperdb.Password): - m = pwre.match(value) - if m: - # password is being given to us encrypted - p = password.Password() - p.scheme = m.group(1) - p.password = m.group(2) - props[propname] = p - else: - props[propname] = password.Password(value) - elif isinstance(proptype, hyperdb.Multilink): - props[propname] = value.split(',') - elif isinstance(proptype, hyperdb.Boolean): - props[propname] = value.lower() in ('yes', 'true', 'on', '1') - elif isinstance(proptype, hyperdb.Number): - props[propname] = int(value) - - # check for the key property - propname = cl.getkey() - if propname and not props.has_key(propname): - raise UsageError, _('you must provide the "%(propname)s" ' - 'property.')%locals() - - # do the actual create - try: - print apply(cl.create, (), props) - except (TypeError, IndexError, ValueError), message: - raise UsageError, message - return 0 - - def do_list(self, args): - '''Usage: list classname [property] - List the instances of a class. - - Lists all instances of the given class. If the property is not - specified, the "label" property is used. The label property is tried - in order: the key, "name", "title" and then the first property, - alphabetically. - ''' - if len(args) < 1: - raise UsageError, _('Not enough arguments supplied') - classname = args[0] - - # get the class - cl = self.get_class(classname) - - # figure the property - if len(args) > 1: - propname = args[1] - else: - propname = cl.labelprop() - - if self.comma_sep: - print ','.join(cl.list()) - else: - for nodeid in cl.list(): - try: - value = cl.get(nodeid, propname) - except KeyError: - raise UsageError, _('%(classname)s has no property ' - '"%(propname)s"')%locals() - print _('%(nodeid)4s: %(value)s')%locals() - return 0 - - def do_table(self, args): - '''Usage: table classname [property[,property]*] - List the instances of a class in tabular form. - - Lists all instances of the given class. If the properties are not - specified, all properties are displayed. By default, the column widths - are the width of the property names. The width may be explicitly defined - by defining the property as "name:width". For example:: - roundup> table priority id,name:10 - Id Name - 1 fatal-bug - 2 bug - 3 usability - 4 feature - ''' - if len(args) < 1: - raise UsageError, _('Not enough arguments supplied') - classname = args[0] - - # get the class - cl = self.get_class(classname) - - # figure the property names to display - if len(args) > 1: - prop_names = args[1].split(',') - all_props = cl.getprops() - for spec in prop_names: - if ':' in spec: - try: - propname, width = spec.split(':') - except (ValueError, TypeError): - raise UsageError, _('"%(spec)s" not name:width')%locals() - else: - propname = spec - if not all_props.has_key(propname): - raise UsageError, _('%(classname)s has no property ' - '"%(propname)s"')%locals() - else: - prop_names = cl.getprops().keys() - - # now figure column widths - props = [] - for spec in prop_names: - if ':' in spec: - name, width = spec.split(':') - props.append((name, int(width))) - else: - props.append((spec, len(spec))) - - # now display the heading - print ' '.join([name.capitalize().ljust(width) for name,width in props]) - - # and the table data - for nodeid in cl.list(): - l = [] - for name, width in props: - if name != 'id': - try: - value = str(cl.get(nodeid, name)) - except KeyError: - # we already checked if the property is valid - a - # KeyError here means the node just doesn't have a - # value for it - value = '' - else: - value = str(nodeid) - f = '%%-%ds'%width - l.append(f%value[:width]) - print ' '.join(l) - return 0 - - def do_history(self, args): - '''Usage: history designator - Show the history entries of a designator. - - Lists the journal entries for the node identified by the designator. - ''' - if len(args) < 1: - raise UsageError, _('Not enough arguments supplied') - try: - classname, nodeid = hyperdb.splitDesignator(args[0]) - except hyperdb.DesignatorError, message: - raise UsageError, message - - try: - print self.db.getclass(classname).history(nodeid) - except KeyError: - raise UsageError, _('no such class "%(classname)s"')%locals() - except IndexError: - raise UsageError, _('no such %(classname)s node "%(nodeid)s"')%locals() - return 0 - - def do_commit(self, args): - '''Usage: commit - Commit all changes made to the database. - - The changes made during an interactive session are not - automatically written to the database - they must be committed - using this command. - - One-off commands on the command-line are automatically committed if - they are successful. - ''' - self.db.commit() - return 0 - - def do_rollback(self, args): - '''Usage: rollback - Undo all changes that are pending commit to the database. - - The changes made during an interactive session are not - automatically written to the database - they must be committed - manually. This command undoes all those changes, so a commit - immediately after would make no changes to the database. - ''' - self.db.rollback() - return 0 - - def do_retire(self, args): - '''Usage: retire designator[,designator]* - Retire the node specified by designator. - - This action indicates that a particular node is not to be retrieved by - the list or find commands, and its key value may be re-used. - ''' - if len(args) < 1: - raise UsageError, _('Not enough arguments supplied') - designators = args[0].split(',') - for designator in designators: - try: - classname, nodeid = hyperdb.splitDesignator(designator) - except hyperdb.DesignatorError, message: - raise UsageError, message - try: - self.db.getclass(classname).retire(nodeid) - except KeyError: - raise UsageError, _('no such class "%(classname)s"')%locals() - except IndexError: - raise UsageError, _('no such %(classname)s node "%(nodeid)s"')%locals() - return 0 - - def do_export(self, args): - '''Usage: export [class[,class]] export_dir - Export the database to colon-separated-value files. - - This action exports the current data from the database into - colon-separated-value files that are placed in the nominated - destination directory. The journals are not exported. - ''' - # we need the CSV module - if csv is None: - raise UsageError, \ - _('Sorry, you need the csv module to use this function.\n' - 'Get it from: http://www.object-craft.com.au/projects/csv/') - - # grab the directory to export to - if len(args) < 1: - raise UsageError, _('Not enough arguments supplied') - dir = args[-1] - - # get the list of classes to export - if len(args) == 2: - classes = args[0].split(',') - else: - classes = self.db.classes.keys() - - # use the csv parser if we can - it's faster - p = csv.parser(field_sep=':') - - # do all the classes specified - for classname in classes: - cl = self.get_class(classname) - f = open(os.path.join(dir, classname+'.csv'), 'w') - properties = cl.getprops() - propnames = properties.keys() - propnames.sort() - print >> f, p.join(propnames) - - # all nodes for this class - for nodeid in cl.list(): - print >>f, p.join(cl.export_list(propnames, nodeid)) - return 0 - - def do_import(self, args): - '''Usage: import import_dir - Import a database from the directory containing CSV files, one per - class to import. - - The files must define the same properties as the class (including having - a "header" line with those property names.) - - The imported nodes will have the same nodeid as defined in the - import file, thus replacing any existing content. - - The new nodes are added to the existing database - if you want to - create a new database using the imported data, then create a new - database (or, tediously, retire all the old data.) - ''' - if len(args) < 1: - raise UsageError, _('Not enough arguments supplied') - if csv is None: - raise UsageError, \ - _('Sorry, you need the csv module to use this function.\n' - 'Get it from: http://www.object-craft.com.au/projects/csv/') - - from roundup import hyperdb - - for file in os.listdir(args[0]): - f = open(os.path.join(args[0], file)) - - # get the classname - classname = os.path.splitext(file)[0] - - # ensure that the properties and the CSV file headings match - cl = self.get_class(classname) - p = csv.parser(field_sep=':') - file_props = p.parse(f.readline()) - properties = cl.getprops() - propnames = properties.keys() - propnames.sort() - m = file_props[:] - m.sort() - if m != propnames: - raise UsageError, _('Import file doesn\'t define the same ' - 'properties as "%(arg0)s".')%{'arg0': args[0]} - - # loop through the file and create a node for each entry - maxid = 1 - while 1: - line = f.readline() - if not line: break - - # parse lines until we get a complete entry - while 1: - l = p.parse(line) - if l: break - line = f.readline() - if not line: - raise ValueError, "Unexpected EOF during CSV parse" - - # do the import and figure the current highest nodeid - maxid = max(maxid, int(cl.import_list(propnames, l))) - - print 'setting', classname, maxid+1 - self.db.setid(classname, str(maxid+1)) - return 0 - - def do_pack(self, args): - '''Usage: pack period | date - -Remove journal entries older than a period of time specified or -before a certain date. - -A period is specified using the suffixes "y", "m", and "d". The -suffix "w" (for "week") means 7 days. - - "3y" means three years - "2y 1m" means two years and one month - "1m 25d" means one month and 25 days - "2w 3d" means two weeks and three days - -Date format is "YYYY-MM-DD" eg: - 2001-01-01 - - ''' - if len(args) <> 1: - raise UsageError, _('Not enough arguments supplied') - - # are we dealing with a period or a date - value = args[0] - date_re = re.compile(r''' - (?P<date>\d\d\d\d-\d\d?-\d\d?)? # yyyy-mm-dd - (?P<period>(\d+y\s*)?(\d+m\s*)?(\d+d\s*)?)? - ''', re.VERBOSE) - m = date_re.match(value) - if not m: - raise ValueError, _('Invalid format') - m = m.groupdict() - if m['period']: - pack_before = date.Date(". - %s"%value) - elif m['date']: - pack_before = date.Date(value) - self.db.pack(pack_before) - return 0 - - def do_reindex(self, args): - '''Usage: reindex - Re-generate a tracker's search indexes. - - This will re-generate the search indexes for a tracker. This will - typically happen automatically. - ''' - self.db.indexer.force_reindex() - self.db.reindex() - return 0 - - def do_security(self, args): - '''Usage: security [Role name] - Display the Permissions available to one or all Roles. - ''' - if len(args) == 1: - role = args[0] - try: - roles = [(args[0], self.db.security.role[args[0]])] - except KeyError: - print _('No such Role "%(role)s"')%locals() - return 1 - else: - roles = self.db.security.role.items() - role = self.db.config.NEW_WEB_USER_ROLES - if ',' in role: - print _('New Web users get the Roles "%(role)s"')%locals() - else: - print _('New Web users get the Role "%(role)s"')%locals() - role = self.db.config.NEW_EMAIL_USER_ROLES - if ',' in role: - print _('New Email users get the Roles "%(role)s"')%locals() - else: - print _('New Email users get the Role "%(role)s"')%locals() - roles.sort() - for rolename, role in roles: - print _('Role "%(name)s":')%role.__dict__ - for permission in role.permissions: - if permission.klass: - print _(' %(description)s (%(name)s for "%(klass)s" ' - 'only)')%permission.__dict__ - else: - print _(' %(description)s (%(name)s)')%permission.__dict__ - return 0 - - def run_command(self, args): - '''Run a single command - ''' - command = args[0] - - # handle help now - if command == 'help': - if len(args)>1: - self.do_help(args[1:]) - return 0 - self.do_help(['help']) - return 0 - if command == 'morehelp': - self.do_help(['help']) - self.help_commands() - self.help_all() - return 0 - - # figure what the command is - try: - functions = self.commands.get(command) - except KeyError: - # not a valid command - print _('Unknown command "%(command)s" ("help commands" for a ' - 'list)')%locals() - return 1 - - # check for multiple matches - if len(functions) > 1: - print _('Multiple commands match "%(command)s": %(list)s')%{'command': - command, 'list': ', '.join([i[0] for i in functions])} - return 1 - command, function = functions[0] - - # make sure we have a tracker_home - while not self.tracker_home: - self.tracker_home = raw_input(_('Enter tracker home: ')).strip() - - # before we open the db, we may be doing an install or init - if command == 'initialise': - try: - return self.do_initialise(self.tracker_home, args) - except UsageError, message: - print _('Error: %(message)s')%locals() - return 1 - elif command == 'install': - try: - return self.do_install(self.tracker_home, args) - except UsageError, message: - print _('Error: %(message)s')%locals() - return 1 - - # get the tracker - try: - tracker = roundup.instance.open(self.tracker_home) - except ValueError, message: - self.tracker_home = '' - print _("Error: Couldn't open tracker: %(message)s")%locals() - return 1 - - # only open the database once! - if not self.db: - self.db = tracker.open('admin') - - # do the command - ret = 0 - try: - ret = function(args[1:]) - except UsageError, message: - print _('Error: %(message)s')%locals() - print - print function.__doc__ - ret = 1 - except: - import traceback - traceback.print_exc() - ret = 1 - return ret - - def interactive(self): - '''Run in an interactive mode - ''' - print _('Roundup %s ready for input.'%roundup_version) - print _('Type "help" for help.') - try: - import readline - except ImportError: - print _('Note: command history and editing not available') - - while 1: - try: - command = raw_input(_('roundup> ')) - except EOFError: - print _('exit...') - break - if not command: continue - args = token.token_split(command) - if not args: continue - if args[0] in ('quit', 'exit'): break - self.run_command(args) - - # exit.. check for transactions - if self.db and self.db.transactions: - commit = raw_input(_('There are unsaved changes. Commit them (y/N)? ')) - if commit and commit[0].lower() == 'y': - self.db.commit() - return 0 - - def main(self): - try: - opts, args = getopt.getopt(sys.argv[1:], 'i:u:hc') - except getopt.GetoptError, e: - self.usage(str(e)) - return 1 - - # handle command-line args - self.tracker_home = os.environ.get('TRACKER_HOME', '') - # TODO: reinstate the user/password stuff (-u arg too) - name = password = '' - if os.environ.has_key('ROUNDUP_LOGIN'): - l = os.environ['ROUNDUP_LOGIN'].split(':') - name = l[0] - if len(l) > 1: - password = l[1] - self.comma_sep = 0 - for opt, arg in opts: - if opt == '-h': - self.usage() - return 0 - if opt == '-i': - self.tracker_home = arg - if opt == '-c': - self.comma_sep = 1 - - # if no command - go interactive - # wrap in a try/finally so we always close off the db - ret = 0 - try: - if not args: - self.interactive() - else: - ret = self.run_command(args) - if self.db: self.db.commit() - return ret - finally: - if self.db: - self.db.close() - -if __name__ == '__main__': - tool = AdminTool() - sys.exit(tool.main()) - -# vim: set filetype=python ts=4 sw=4 et si
--- a/roundup/backends/__init__.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,85 +0,0 @@ -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: __init__.py,v 1.18 2002-09-23 12:02:53 richard Exp $ - -__all__ = [] - -try: - import sys, anydbm - if not hasattr(sys, 'version_info') or sys.version_info < (2,1,2): - import dumbdbm - # dumbdbm only works in python 2.1.2+ - assert anydbm._defaultmod != dumbdbm - del anydbm - del dumbdbm -except AssertionError: - print "WARNING: you should upgrade to python 2.1.3" -except ImportError, message: - if str(message) != 'No module named anydbm': raise -else: - import back_anydbm - anydbm = back_anydbm - __all__.append('anydbm') - -try: - import gadfly - import gadfly.client -except ImportError, message: - if str(message) != 'No module named gadfly': raise -else: - import back_gadfly - gadfly = back_gadfly - __all__.append('gadfly') - -try: - import sqlite -except ImportError, message: - if str(message) != 'No module named sqlite': raise -else: - import back_sqlite - sqlite = back_sqlite - __all__.append('sqlite') - -try: - import bsddb -except ImportError, message: - if str(message) != 'No module named bsddb': raise -else: - import back_bsddb - bsddb = back_bsddb - __all__.append('bsddb') - -try: - import bsddb3 -except ImportError, message: - if str(message) != 'No module named bsddb3': raise -else: - import back_bsddb3 - bsddb3 = back_bsddb3 - __all__.append('bsddb3') - -try: - import metakit -except ImportError, message: - if str(message) != 'No module named metakit': raise -else: - import back_metakit - metakit = back_metakit - __all__.append('metakit') - -# vim: set filetype=python ts=4 sw=4 et si
--- a/roundup/backends/back_anydbm.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1972 +0,0 @@ -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -#$Id: back_anydbm.py,v 1.86 2002-09-26 03:04:24 richard Exp $ -''' -This module defines a backend that saves the hyperdatabase in a database -chosen by anydbm. It is guaranteed to always be available in python -versions >2.1.1 (the dumbdbm fallback in 2.1.1 and earlier has several -serious bugs, and is not available) -''' - -import whichdb, anydbm, os, marshal, re, weakref, string, copy -from roundup import hyperdb, date, password, roundupdb, security -from blobfiles import FileStorage -from sessions import Sessions -from roundup.indexer import Indexer -from locking import acquire_lock, release_lock -from roundup.hyperdb import String, Password, Date, Interval, Link, \ - Multilink, DatabaseError, Boolean, Number - -# -# Now the database -# -class Database(FileStorage, hyperdb.Database, roundupdb.Database): - '''A database for storing records containing flexible data types. - - Transaction stuff TODO: - . check the timestamp of the class file and nuke the cache if it's - modified. Do some sort of conflict checking on the dirty stuff. - . perhaps detect write collisions (related to above)? - - ''' - def __init__(self, config, journaltag=None): - '''Open a hyperdatabase given a specifier to some storage. - - The 'storagelocator' is obtained from config.DATABASE. - The meaning of 'storagelocator' depends on the particular - implementation of the hyperdatabase. It could be a file name, - a directory path, a socket descriptor for a connection to a - database over the network, etc. - - The 'journaltag' is a token that will be attached to the journal - entries for any edits done on the database. If 'journaltag' is - None, the database is opened in read-only mode: the Class.create(), - Class.set(), and Class.retire() methods are disabled. - ''' - self.config, self.journaltag = config, journaltag - self.dir = config.DATABASE - self.classes = {} - self.cache = {} # cache of nodes loaded or created - self.dirtynodes = {} # keep track of the dirty nodes by class - self.newnodes = {} # keep track of the new nodes by class - self.destroyednodes = {}# keep track of the destroyed nodes by class - self.transactions = [] - self.indexer = Indexer(self.dir) - self.sessions = Sessions(self.config) - self.security = security.Security(self) - # ensure files are group readable and writable - os.umask(0002) - - def post_init(self): - ''' Called once the schema initialisation has finished. - ''' - # reindex the db if necessary - if self.indexer.should_reindex(): - self.reindex() - - # figure the "curuserid" - if self.journaltag is None: - self.curuserid = None - elif self.journaltag == 'admin': - # admin user may not exist, but always has ID 1 - self.curuserid = '1' - else: - self.curuserid = self.user.lookup(self.journaltag) - - def reindex(self): - for klass in self.classes.values(): - for nodeid in klass.list(): - klass.index(nodeid) - self.indexer.save_index() - - def __repr__(self): - return '<back_anydbm instance at %x>'%id(self) - - # - # Classes - # - def __getattr__(self, classname): - '''A convenient way of calling self.getclass(classname).''' - if self.classes.has_key(classname): - if __debug__: - print >>hyperdb.DEBUG, '__getattr__', (self, classname) - return self.classes[classname] - raise AttributeError, classname - - def addclass(self, cl): - if __debug__: - print >>hyperdb.DEBUG, 'addclass', (self, cl) - cn = cl.classname - if self.classes.has_key(cn): - raise ValueError, cn - self.classes[cn] = cl - - def getclasses(self): - '''Return a list of the names of all existing classes.''' - if __debug__: - print >>hyperdb.DEBUG, 'getclasses', (self,) - l = self.classes.keys() - l.sort() - return l - - def getclass(self, classname): - '''Get the Class object representing a particular class. - - If 'classname' is not a valid class name, a KeyError is raised. - ''' - if __debug__: - print >>hyperdb.DEBUG, 'getclass', (self, classname) - try: - return self.classes[classname] - except KeyError: - raise KeyError, 'There is no class called "%s"'%classname - - # - # Class DBs - # - def clear(self): - '''Delete all database contents - ''' - if __debug__: - print >>hyperdb.DEBUG, 'clear', (self,) - for cn in self.classes.keys(): - for dummy in 'nodes', 'journals': - path = os.path.join(self.dir, 'journals.%s'%cn) - if os.path.exists(path): - os.remove(path) - elif os.path.exists(path+'.db'): # dbm appends .db - os.remove(path+'.db') - - def getclassdb(self, classname, mode='r'): - ''' grab a connection to the class db that will be used for - multiple actions - ''' - if __debug__: - print >>hyperdb.DEBUG, 'getclassdb', (self, classname, mode) - return self.opendb('nodes.%s'%classname, mode) - - def determine_db_type(self, path): - ''' determine which DB wrote the class file - ''' - db_type = '' - if os.path.exists(path): - db_type = whichdb.whichdb(path) - if not db_type: - raise DatabaseError, "Couldn't identify database type" - elif os.path.exists(path+'.db'): - # if the path ends in '.db', it's a dbm database, whether - # anydbm says it's dbhash or not! - db_type = 'dbm' - return db_type - - def opendb(self, name, mode): - '''Low-level database opener that gets around anydbm/dbm - eccentricities. - ''' - if __debug__: - print >>hyperdb.DEBUG, 'opendb', (self, name, mode) - - # figure the class db type - path = os.path.join(os.getcwd(), self.dir, name) - db_type = self.determine_db_type(path) - - # new database? let anydbm pick the best dbm - if not db_type: - if __debug__: - print >>hyperdb.DEBUG, "opendb anydbm.open(%r, 'c')"%path - return anydbm.open(path, 'c') - - # open the database with the correct module - try: - dbm = __import__(db_type) - except ImportError: - raise DatabaseError, \ - "Couldn't open database - the required module '%s'"\ - " is not available"%db_type - if __debug__: - print >>hyperdb.DEBUG, "opendb %r.open(%r, %r)"%(db_type, path, - mode) - return dbm.open(path, mode) - - def lockdb(self, name): - ''' Lock a database file - ''' - path = os.path.join(os.getcwd(), self.dir, '%s.lock'%name) - return acquire_lock(path) - - # - # Node IDs - # - def newid(self, classname): - ''' Generate a new id for the given class - ''' - # open the ids DB - create if if doesn't exist - lock = self.lockdb('_ids') - db = self.opendb('_ids', 'c') - if db.has_key(classname): - newid = db[classname] = str(int(db[classname]) + 1) - else: - # the count() bit is transitional - older dbs won't start at 1 - newid = str(self.getclass(classname).count()+1) - db[classname] = newid - db.close() - release_lock(lock) - return newid - - def setid(self, classname, setid): - ''' Set the id counter: used during import of database - ''' - # open the ids DB - create if if doesn't exist - lock = self.lockdb('_ids') - db = self.opendb('_ids', 'c') - db[classname] = str(setid) - db.close() - release_lock(lock) - - # - # Nodes - # - def addnode(self, classname, nodeid, node): - ''' add the specified node to its class's db - ''' - if __debug__: - print >>hyperdb.DEBUG, 'addnode', (self, classname, nodeid, node) - - # we'll be supplied these props if we're doing an import - if not node.has_key('creator'): - # add in the "calculated" properties (dupe so we don't affect - # calling code's node assumptions) - node = node.copy() - node['creator'] = self.curuserid - node['creation'] = node['activity'] = date.Date() - - self.newnodes.setdefault(classname, {})[nodeid] = 1 - self.cache.setdefault(classname, {})[nodeid] = node - self.savenode(classname, nodeid, node) - - def setnode(self, classname, nodeid, node): - ''' change the specified node - ''' - if __debug__: - print >>hyperdb.DEBUG, 'setnode', (self, classname, nodeid, node) - self.dirtynodes.setdefault(classname, {})[nodeid] = 1 - - # update the activity time (dupe so we don't affect - # calling code's node assumptions) - node = node.copy() - node['activity'] = date.Date() - - # can't set without having already loaded the node - self.cache[classname][nodeid] = node - self.savenode(classname, nodeid, node) - - def savenode(self, classname, nodeid, node): - ''' perform the saving of data specified by the set/addnode - ''' - if __debug__: - print >>hyperdb.DEBUG, 'savenode', (self, classname, nodeid, node) - self.transactions.append((self.doSaveNode, (classname, nodeid, node))) - - def getnode(self, classname, nodeid, db=None, cache=1): - ''' get a node from the database - ''' - if __debug__: - print >>hyperdb.DEBUG, 'getnode', (self, classname, nodeid, db) - if cache: - # try the cache - cache_dict = self.cache.setdefault(classname, {}) - if cache_dict.has_key(nodeid): - if __debug__: - print >>hyperdb.TRACE, 'get %s %s cached'%(classname, - nodeid) - return cache_dict[nodeid] - - if __debug__: - print >>hyperdb.TRACE, 'get %s %s'%(classname, nodeid) - - # get from the database and save in the cache - if db is None: - db = self.getclassdb(classname) - if not db.has_key(nodeid): - raise IndexError, "no such %s %s"%(classname, nodeid) - - # check the uncommitted, destroyed nodes - if (self.destroyednodes.has_key(classname) and - self.destroyednodes[classname].has_key(nodeid)): - raise IndexError, "no such %s %s"%(classname, nodeid) - - # decode - res = marshal.loads(db[nodeid]) - - # reverse the serialisation - res = self.unserialise(classname, res) - - # store off in the cache dict - if cache: - cache_dict[nodeid] = res - - return res - - def destroynode(self, classname, nodeid): - '''Remove a node from the database. Called exclusively by the - destroy() method on Class. - ''' - if __debug__: - print >>hyperdb.DEBUG, 'destroynode', (self, classname, nodeid) - - # remove from cache and newnodes if it's there - if (self.cache.has_key(classname) and - self.cache[classname].has_key(nodeid)): - del self.cache[classname][nodeid] - if (self.newnodes.has_key(classname) and - self.newnodes[classname].has_key(nodeid)): - del self.newnodes[classname][nodeid] - - # see if there's any obvious commit actions that we should get rid of - for entry in self.transactions[:]: - if entry[1][:2] == (classname, nodeid): - self.transactions.remove(entry) - - # add to the destroyednodes map - self.destroyednodes.setdefault(classname, {})[nodeid] = 1 - - # add the destroy commit action - self.transactions.append((self.doDestroyNode, (classname, nodeid))) - - def serialise(self, classname, node): - '''Copy the node contents, converting non-marshallable data into - marshallable data. - ''' - if __debug__: - print >>hyperdb.DEBUG, 'serialise', classname, node - properties = self.getclass(classname).getprops() - d = {} - for k, v in node.items(): - # if the property doesn't exist, or is the "retired" flag then - # it won't be in the properties dict - if not properties.has_key(k): - d[k] = v - continue - - # get the property spec - prop = properties[k] - - if isinstance(prop, Password): - d[k] = str(v) - elif isinstance(prop, Date) and v is not None: - d[k] = v.serialise() - elif isinstance(prop, Interval) and v is not None: - d[k] = v.serialise() - else: - d[k] = v - return d - - def unserialise(self, classname, node): - '''Decode the marshalled node data - ''' - if __debug__: - print >>hyperdb.DEBUG, 'unserialise', classname, node - properties = self.getclass(classname).getprops() - d = {} - for k, v in node.items(): - # if the property doesn't exist, or is the "retired" flag then - # it won't be in the properties dict - if not properties.has_key(k): - d[k] = v - continue - - # get the property spec - prop = properties[k] - - if isinstance(prop, Date) and v is not None: - d[k] = date.Date(v) - elif isinstance(prop, Interval) and v is not None: - d[k] = date.Interval(v) - elif isinstance(prop, Password): - p = password.Password() - p.unpack(v) - d[k] = p - else: - d[k] = v - return d - - def hasnode(self, classname, nodeid, db=None): - ''' determine if the database has a given node - ''' - if __debug__: - print >>hyperdb.DEBUG, 'hasnode', (self, classname, nodeid, db) - - # try the cache - cache = self.cache.setdefault(classname, {}) - if cache.has_key(nodeid): - if __debug__: - print >>hyperdb.TRACE, 'has %s %s cached'%(classname, nodeid) - return 1 - if __debug__: - print >>hyperdb.TRACE, 'has %s %s'%(classname, nodeid) - - # not in the cache - check the database - if db is None: - db = self.getclassdb(classname) - res = db.has_key(nodeid) - return res - - def countnodes(self, classname, db=None): - if __debug__: - print >>hyperdb.DEBUG, 'countnodes', (self, classname, db) - - count = 0 - - # include the uncommitted nodes - if self.newnodes.has_key(classname): - count += len(self.newnodes[classname]) - if self.destroyednodes.has_key(classname): - count -= len(self.destroyednodes[classname]) - - # and count those in the DB - if db is None: - db = self.getclassdb(classname) - count = count + len(db.keys()) - return count - - def getnodeids(self, classname, db=None): - if __debug__: - print >>hyperdb.DEBUG, 'getnodeids', (self, classname, db) - - res = [] - - # start off with the new nodes - if self.newnodes.has_key(classname): - res += self.newnodes[classname].keys() - - if db is None: - db = self.getclassdb(classname) - res = res + db.keys() - - # remove the uncommitted, destroyed nodes - if self.destroyednodes.has_key(classname): - for nodeid in self.destroyednodes[classname].keys(): - if db.has_key(nodeid): - res.remove(nodeid) - - return res - - - # - # Files - special node properties - # inherited from FileStorage - - # - # Journal - # - def addjournal(self, classname, nodeid, action, params, creator=None, - creation=None): - ''' Journal the Action - 'action' may be: - - 'create' or 'set' -- 'params' is a dictionary of property values - 'link' or 'unlink' -- 'params' is (classname, nodeid, propname) - 'retire' -- 'params' is None - ''' - if __debug__: - print >>hyperdb.DEBUG, 'addjournal', (self, classname, nodeid, - action, params, creator, creation) - self.transactions.append((self.doSaveJournal, (classname, nodeid, - action, params, creator, creation))) - - def getjournal(self, classname, nodeid): - ''' get the journal for id - - Raise IndexError if the node doesn't exist (as per history()'s - API) - ''' - if __debug__: - print >>hyperdb.DEBUG, 'getjournal', (self, classname, nodeid) - # attempt to open the journal - in some rare cases, the journal may - # not exist - try: - db = self.opendb('journals.%s'%classname, 'r') - except anydbm.error, error: - if str(error) == "need 'c' or 'n' flag to open new db": - raise IndexError, 'no such %s %s'%(classname, nodeid) - elif error.args[0] != 2: - raise - raise IndexError, 'no such %s %s'%(classname, nodeid) - try: - journal = marshal.loads(db[nodeid]) - except KeyError: - db.close() - raise IndexError, 'no such %s %s'%(classname, nodeid) - db.close() - res = [] - for nodeid, date_stamp, user, action, params in journal: - res.append((nodeid, date.Date(date_stamp), user, action, params)) - return res - - def pack(self, pack_before): - ''' Delete all journal entries except "create" before 'pack_before'. - ''' - if __debug__: - print >>hyperdb.DEBUG, 'packjournal', (self, pack_before) - - pack_before = pack_before.serialise() - for classname in self.getclasses(): - # get the journal db - db_name = 'journals.%s'%classname - path = os.path.join(os.getcwd(), self.dir, classname) - db_type = self.determine_db_type(path) - db = self.opendb(db_name, 'w') - - for key in db.keys(): - # get the journal for this db entry - journal = marshal.loads(db[key]) - l = [] - last_set_entry = None - for entry in journal: - # unpack the entry - (nodeid, date_stamp, self.journaltag, action, - params) = entry - # if the entry is after the pack date, _or_ the initial - # create entry, then it stays - if date_stamp > pack_before or action == 'create': - l.append(entry) - db[key] = marshal.dumps(l) - if db_type == 'gdbm': - db.reorganize() - db.close() - - - # - # Basic transaction support - # - def commit(self): - ''' Commit the current transactions. - ''' - if __debug__: - print >>hyperdb.DEBUG, 'commit', (self,) - # TODO: lock the DB - - # keep a handle to all the database files opened - self.databases = {} - - # now, do all the transactions - reindex = {} - for method, args in self.transactions: - reindex[method(*args)] = 1 - - # now close all the database files - for db in self.databases.values(): - db.close() - del self.databases - # TODO: unlock the DB - - # reindex the nodes that request it - for classname, nodeid in filter(None, reindex.keys()): - print >>hyperdb.DEBUG, 'commit.reindex', (classname, nodeid) - self.getclass(classname).index(nodeid) - - # save the indexer state - self.indexer.save_index() - - self.clearCache() - - def clearCache(self): - # all transactions committed, back to normal - self.cache = {} - self.dirtynodes = {} - self.newnodes = {} - self.destroyednodes = {} - self.transactions = [] - - def getCachedClassDB(self, classname): - ''' get the class db, looking in our cache of databases for commit - ''' - # get the database handle - db_name = 'nodes.%s'%classname - if not self.databases.has_key(db_name): - self.databases[db_name] = self.getclassdb(classname, 'c') - return self.databases[db_name] - - def doSaveNode(self, classname, nodeid, node): - if __debug__: - print >>hyperdb.DEBUG, 'doSaveNode', (self, classname, nodeid, - node) - - db = self.getCachedClassDB(classname) - - # now save the marshalled data - db[nodeid] = marshal.dumps(self.serialise(classname, node)) - - # return the classname, nodeid so we reindex this content - return (classname, nodeid) - - def getCachedJournalDB(self, classname): - ''' get the journal db, looking in our cache of databases for commit - ''' - # get the database handle - db_name = 'journals.%s'%classname - if not self.databases.has_key(db_name): - self.databases[db_name] = self.opendb(db_name, 'c') - return self.databases[db_name] - - def doSaveJournal(self, classname, nodeid, action, params, creator, - creation): - # serialise the parameters now if necessary - if isinstance(params, type({})): - if action in ('set', 'create'): - params = self.serialise(classname, params) - - # handle supply of the special journalling parameters (usually - # supplied on importing an existing database) - if creator: - journaltag = creator - else: - journaltag = self.curuserid - if creation: - journaldate = creation.serialise() - else: - journaldate = date.Date().serialise() - - # create the journal entry - entry = (nodeid, journaldate, journaltag, action, params) - - if __debug__: - print >>hyperdb.DEBUG, 'doSaveJournal', entry - - db = self.getCachedJournalDB(classname) - - # now insert the journal entry - if db.has_key(nodeid): - # append to existing - s = db[nodeid] - l = marshal.loads(s) - l.append(entry) - else: - l = [entry] - - db[nodeid] = marshal.dumps(l) - - def doDestroyNode(self, classname, nodeid): - if __debug__: - print >>hyperdb.DEBUG, 'doDestroyNode', (self, classname, nodeid) - - # delete from the class database - db = self.getCachedClassDB(classname) - if db.has_key(nodeid): - del db[nodeid] - - # delete from the database - db = self.getCachedJournalDB(classname) - if db.has_key(nodeid): - del db[nodeid] - - # return the classname, nodeid so we reindex this content - return (classname, nodeid) - - def rollback(self): - ''' Reverse all actions from the current transaction. - ''' - if __debug__: - print >>hyperdb.DEBUG, 'rollback', (self, ) - for method, args in self.transactions: - # delete temporary files - if method == self.doStoreFile: - self.rollbackStoreFile(*args) - self.cache = {} - self.dirtynodes = {} - self.newnodes = {} - self.destroyednodes = {} - self.transactions = [] - - def close(self): - ''' Nothing to do - ''' - pass - -_marker = [] -class Class(hyperdb.Class): - '''The handle to a particular class of nodes in a hyperdatabase.''' - - def __init__(self, db, classname, **properties): - '''Create a new class with a given name and property specification. - - 'classname' must not collide with the name of an existing class, - or a ValueError is raised. The keyword arguments in 'properties' - must map names to property objects, or a TypeError is raised. - ''' - if (properties.has_key('creation') or properties.has_key('activity') - or properties.has_key('creator')): - raise ValueError, '"creation", "activity" and "creator" are '\ - 'reserved' - - self.classname = classname - self.properties = properties - self.db = weakref.proxy(db) # use a weak ref to avoid circularity - self.key = '' - - # should we journal changes (default yes) - self.do_journal = 1 - - # do the db-related init stuff - db.addclass(self) - - self.auditors = {'create': [], 'set': [], 'retire': []} - self.reactors = {'create': [], 'set': [], 'retire': []} - - def enableJournalling(self): - '''Turn journalling on for this class - ''' - self.do_journal = 1 - - def disableJournalling(self): - '''Turn journalling off for this class - ''' - self.do_journal = 0 - - # Editing nodes: - - def create(self, **propvalues): - '''Create a new node of this class and return its id. - - The keyword arguments in 'propvalues' map property names to values. - - The values of arguments must be acceptable for the types of their - corresponding properties or a TypeError is raised. - - If this class has a key property, it must be present and its value - must not collide with other key strings or a ValueError is raised. - - Any other properties on this class that are missing from the - 'propvalues' dictionary are set to None. - - If an id in a link or multilink property does not refer to a valid - node, an IndexError is raised. - - These operations trigger detectors and can be vetoed. Attempts - to modify the "creation" or "activity" properties cause a KeyError. - ''' - if propvalues.has_key('id'): - raise KeyError, '"id" is reserved' - - if self.db.journaltag is None: - raise DatabaseError, 'Database open read-only' - - if propvalues.has_key('creation') or propvalues.has_key('activity'): - raise KeyError, '"creation" and "activity" are reserved' - - self.fireAuditors('create', None, propvalues) - - # new node's id - newid = self.db.newid(self.classname) - - # validate propvalues - num_re = re.compile('^\d+$') - for key, value in propvalues.items(): - if key == self.key: - try: - self.lookup(value) - except KeyError: - pass - else: - raise ValueError, 'node with key "%s" exists'%value - - # try to handle this property - try: - prop = self.properties[key] - except KeyError: - raise KeyError, '"%s" has no property "%s"'%(self.classname, - key) - - if value is not None and isinstance(prop, Link): - if type(value) != type(''): - raise ValueError, 'link value must be String' - link_class = self.properties[key].classname - # if it isn't a number, it's a key - if not num_re.match(value): - try: - value = self.db.classes[link_class].lookup(value) - except (TypeError, KeyError): - raise IndexError, 'new property "%s": %s not a %s'%( - key, value, link_class) - elif not self.db.getclass(link_class).hasnode(value): - raise IndexError, '%s has no node %s'%(link_class, value) - - # save off the value - propvalues[key] = value - - # register the link with the newly linked node - if self.do_journal and self.properties[key].do_journal: - self.db.addjournal(link_class, value, 'link', - (self.classname, newid, key)) - - elif isinstance(prop, Multilink): - if type(value) != type([]): - raise TypeError, 'new property "%s" not a list of ids'%key - - # clean up and validate the list of links - link_class = self.properties[key].classname - l = [] - for entry in value: - if type(entry) != type(''): - raise ValueError, '"%s" multilink value (%r) '\ - 'must contain Strings'%(key, value) - # if it isn't a number, it's a key - if not num_re.match(entry): - try: - entry = self.db.classes[link_class].lookup(entry) - except (TypeError, KeyError): - raise IndexError, 'new property "%s": %s not a %s'%( - key, entry, self.properties[key].classname) - l.append(entry) - value = l - propvalues[key] = value - - # handle additions - for nodeid in value: - if not self.db.getclass(link_class).hasnode(nodeid): - raise IndexError, '%s has no node %s'%(link_class, - nodeid) - # register the link with the newly linked node - if self.do_journal and self.properties[key].do_journal: - self.db.addjournal(link_class, nodeid, 'link', - (self.classname, newid, key)) - - elif isinstance(prop, String): - if type(value) != type(''): - raise TypeError, 'new property "%s" not a string'%key - - elif isinstance(prop, Password): - if not isinstance(value, password.Password): - raise TypeError, 'new property "%s" not a Password'%key - - elif isinstance(prop, Date): - if value is not None and not isinstance(value, date.Date): - raise TypeError, 'new property "%s" not a Date'%key - - elif isinstance(prop, Interval): - if value is not None and not isinstance(value, date.Interval): - raise TypeError, 'new property "%s" not an Interval'%key - - elif value is not None and isinstance(prop, Number): - try: - float(value) - except ValueError: - raise TypeError, 'new property "%s" not numeric'%key - - elif value is not None and isinstance(prop, Boolean): - try: - int(value) - except ValueError: - raise TypeError, 'new property "%s" not boolean'%key - - # make sure there's data where there needs to be - for key, prop in self.properties.items(): - if propvalues.has_key(key): - continue - if key == self.key: - raise ValueError, 'key property "%s" is required'%key - if isinstance(prop, Multilink): - propvalues[key] = [] - else: - propvalues[key] = None - - # done - self.db.addnode(self.classname, newid, propvalues) - if self.do_journal: - self.db.addjournal(self.classname, newid, 'create', propvalues) - - self.fireReactors('create', newid, None) - - return newid - - def export_list(self, propnames, nodeid): - ''' Export a node - generate a list of CSV-able data in the order - specified by propnames for the given node. - ''' - properties = self.getprops() - l = [] - for prop in propnames: - proptype = properties[prop] - value = self.get(nodeid, prop) - # "marshal" data where needed - if value is None: - pass - elif isinstance(proptype, hyperdb.Date): - value = value.get_tuple() - elif isinstance(proptype, hyperdb.Interval): - value = value.get_tuple() - elif isinstance(proptype, hyperdb.Password): - value = str(value) - l.append(repr(value)) - return l - - def import_list(self, propnames, proplist): - ''' Import a node - all information including "id" is present and - should not be sanity checked. Triggers are not triggered. The - journal should be initialised using the "creator" and "created" - information. - - Return the nodeid of the node imported. - ''' - if self.db.journaltag is None: - raise DatabaseError, 'Database open read-only' - properties = self.getprops() - - # make the new node's property map - d = {} - for i in range(len(propnames)): - # Use eval to reverse the repr() used to output the CSV - value = eval(proplist[i]) - - # Figure the property for this column - propname = propnames[i] - prop = properties[propname] - - # "unmarshal" where necessary - if propname == 'id': - newid = value - continue - elif value is None: - # don't set Nones - continue - elif isinstance(prop, hyperdb.Date): - value = date.Date(value) - elif isinstance(prop, hyperdb.Interval): - value = date.Interval(value) - elif isinstance(prop, hyperdb.Password): - pwd = password.Password() - pwd.unpack(value) - value = pwd - d[propname] = value - - # add the node and journal - self.db.addnode(self.classname, newid, d) - - # extract the journalling stuff and nuke it - if d.has_key('creator'): - creator = d['creator'] - del d['creator'] - else: - creator = None - if d.has_key('creation'): - creation = d['creation'] - del d['creation'] - else: - creation = None - if d.has_key('activity'): - del d['activity'] - self.db.addjournal(self.classname, newid, 'create', d, creator, - creation) - return newid - - def get(self, nodeid, propname, default=_marker, cache=1): - '''Get the value of a property on an existing node of this class. - - 'nodeid' must be the id of an existing node of this class or an - IndexError is raised. 'propname' must be the name of a property - of this class or a KeyError is raised. - - 'cache' indicates whether the transaction cache should be queried - for the node. If the node has been modified and you need to - determine what its values prior to modification are, you need to - set cache=0. - - Attempts to get the "creation" or "activity" properties should - do the right thing. - ''' - if propname == 'id': - return nodeid - - # get the node's dict - d = self.db.getnode(self.classname, nodeid, cache=cache) - - # check for one of the special props - if propname == 'creation': - if d.has_key('creation'): - return d['creation'] - if not self.do_journal: - raise ValueError, 'Journalling is disabled for this class' - journal = self.db.getjournal(self.classname, nodeid) - if journal: - return self.db.getjournal(self.classname, nodeid)[0][1] - else: - # on the strange chance that there's no journal - return date.Date() - if propname == 'activity': - if d.has_key('activity'): - return d['activity'] - if not self.do_journal: - raise ValueError, 'Journalling is disabled for this class' - journal = self.db.getjournal(self.classname, nodeid) - if journal: - return self.db.getjournal(self.classname, nodeid)[-1][1] - else: - # on the strange chance that there's no journal - return date.Date() - if propname == 'creator': - if d.has_key('creator'): - return d['creator'] - if not self.do_journal: - raise ValueError, 'Journalling is disabled for this class' - journal = self.db.getjournal(self.classname, nodeid) - if journal: - num_re = re.compile('^\d+$') - value = self.db.getjournal(self.classname, nodeid)[0][2] - if num_re.match(value): - return value - else: - # old-style "username" journal tag - try: - return self.db.user.lookup(value) - except KeyError: - # user's been retired, return admin - return '1' - else: - return self.db.curuserid - - # get the property (raises KeyErorr if invalid) - prop = self.properties[propname] - - if not d.has_key(propname): - if default is _marker: - if isinstance(prop, Multilink): - return [] - else: - return None - else: - return default - - # return a dupe of the list so code doesn't get confused - if isinstance(prop, Multilink): - return d[propname][:] - - return d[propname] - - # not in spec - def getnode(self, nodeid, cache=1): - ''' Return a convenience wrapper for the node. - - 'nodeid' must be the id of an existing node of this class or an - IndexError is raised. - - 'cache' indicates whether the transaction cache should be queried - for the node. If the node has been modified and you need to - determine what its values prior to modification are, you need to - set cache=0. - ''' - return Node(self, nodeid, cache=cache) - - def set(self, nodeid, **propvalues): - '''Modify a property on an existing node of this class. - - 'nodeid' must be the id of an existing node of this class or an - IndexError is raised. - - Each key in 'propvalues' must be the name of a property of this - class or a KeyError is raised. - - All values in 'propvalues' must be acceptable types for their - corresponding properties or a TypeError is raised. - - If the value of the key property is set, it must not collide with - other key strings or a ValueError is raised. - - If the value of a Link or Multilink property contains an invalid - node id, a ValueError is raised. - - These operations trigger detectors and can be vetoed. Attempts - to modify the "creation" or "activity" properties cause a KeyError. - ''' - if not propvalues: - return propvalues - - if propvalues.has_key('creation') or propvalues.has_key('activity'): - raise KeyError, '"creation" and "activity" are reserved' - - if propvalues.has_key('id'): - raise KeyError, '"id" is reserved' - - if self.db.journaltag is None: - raise DatabaseError, 'Database open read-only' - - self.fireAuditors('set', nodeid, propvalues) - # Take a copy of the node dict so that the subsequent set - # operation doesn't modify the oldvalues structure. - try: - # try not using the cache initially - oldvalues = copy.deepcopy(self.db.getnode(self.classname, nodeid, - cache=0)) - except IndexError: - # this will be needed if somone does a create() and set() - # with no intervening commit() - oldvalues = copy.deepcopy(self.db.getnode(self.classname, nodeid)) - - node = self.db.getnode(self.classname, nodeid) - if node.has_key(self.db.RETIRED_FLAG): - raise IndexError - num_re = re.compile('^\d+$') - - # if the journal value is to be different, store it in here - journalvalues = {} - - for propname, value in propvalues.items(): - # check to make sure we're not duplicating an existing key - if propname == self.key and node[propname] != value: - try: - self.lookup(value) - except KeyError: - pass - else: - raise ValueError, 'node with key "%s" exists'%value - - # this will raise the KeyError if the property isn't valid - # ... we don't use getprops() here because we only care about - # the writeable properties. - try: - prop = self.properties[propname] - except KeyError: - raise KeyError, '"%s" has no property named "%s"'%( - self.classname, propname) - - # if the value's the same as the existing value, no sense in - # doing anything - if node.has_key(propname) and value == node[propname]: - del propvalues[propname] - continue - - # do stuff based on the prop type - if isinstance(prop, Link): - link_class = prop.classname - # if it isn't a number, it's a key - if value is not None and not isinstance(value, type('')): - raise ValueError, 'property "%s" link value be a string'%( - propname) - if isinstance(value, type('')) and not num_re.match(value): - try: - value = self.db.classes[link_class].lookup(value) - except (TypeError, KeyError): - raise IndexError, 'new property "%s": %s not a %s'%( - propname, value, prop.classname) - - if (value is not None and - not self.db.getclass(link_class).hasnode(value)): - raise IndexError, '%s has no node %s'%(link_class, value) - - if self.do_journal and prop.do_journal: - # register the unlink with the old linked node - if node[propname] is not None: - self.db.addjournal(link_class, node[propname], 'unlink', - (self.classname, nodeid, propname)) - - # register the link with the newly linked node - if value is not None: - self.db.addjournal(link_class, value, 'link', - (self.classname, nodeid, propname)) - - elif isinstance(prop, Multilink): - if type(value) != type([]): - raise TypeError, 'new property "%s" not a list of'\ - ' ids'%propname - link_class = self.properties[propname].classname - l = [] - for entry in value: - # if it isn't a number, it's a key - if type(entry) != type(''): - raise ValueError, 'new property "%s" link value ' \ - 'must be a string'%propname - if not num_re.match(entry): - try: - entry = self.db.classes[link_class].lookup(entry) - except (TypeError, KeyError): - raise IndexError, 'new property "%s": %s not a %s'%( - propname, entry, - self.properties[propname].classname) - l.append(entry) - value = l - propvalues[propname] = value - - # figure the journal entry for this property - add = [] - remove = [] - - # handle removals - if node.has_key(propname): - l = node[propname] - else: - l = [] - for id in l[:]: - if id in value: - continue - # register the unlink with the old linked node - if self.do_journal and self.properties[propname].do_journal: - self.db.addjournal(link_class, id, 'unlink', - (self.classname, nodeid, propname)) - l.remove(id) - remove.append(id) - - # handle additions - for id in value: - if not self.db.getclass(link_class).hasnode(id): - raise IndexError, '%s has no node %s'%(link_class, id) - if id in l: - continue - # register the link with the newly linked node - if self.do_journal and self.properties[propname].do_journal: - self.db.addjournal(link_class, id, 'link', - (self.classname, nodeid, propname)) - l.append(id) - add.append(id) - - # figure the journal entry - l = [] - if add: - l.append(('+', add)) - if remove: - l.append(('-', remove)) - if l: - journalvalues[propname] = tuple(l) - - elif isinstance(prop, String): - if value is not None and type(value) != type(''): - raise TypeError, 'new property "%s" not a string'%propname - - elif isinstance(prop, Password): - if not isinstance(value, password.Password): - raise TypeError, 'new property "%s" not a Password'%propname - propvalues[propname] = value - - elif value is not None and isinstance(prop, Date): - if not isinstance(value, date.Date): - raise TypeError, 'new property "%s" not a Date'% propname - propvalues[propname] = value - - elif value is not None and isinstance(prop, Interval): - if not isinstance(value, date.Interval): - raise TypeError, 'new property "%s" not an '\ - 'Interval'%propname - propvalues[propname] = value - - elif value is not None and isinstance(prop, Number): - try: - float(value) - except ValueError: - raise TypeError, 'new property "%s" not numeric'%propname - - elif value is not None and isinstance(prop, Boolean): - try: - int(value) - except ValueError: - raise TypeError, 'new property "%s" not boolean'%propname - - node[propname] = value - - # nothing to do? - if not propvalues: - return propvalues - - # do the set, and journal it - self.db.setnode(self.classname, nodeid, node) - - if self.do_journal: - propvalues.update(journalvalues) - self.db.addjournal(self.classname, nodeid, 'set', propvalues) - - self.fireReactors('set', nodeid, oldvalues) - - return propvalues - - def retire(self, nodeid): - '''Retire a node. - - The properties on the node remain available from the get() method, - and the node's id is never reused. - - Retired nodes are not returned by the find(), list(), or lookup() - methods, and other nodes may reuse the values of their key properties. - - These operations trigger detectors and can be vetoed. Attempts - to modify the "creation" or "activity" properties cause a KeyError. - ''' - if self.db.journaltag is None: - raise DatabaseError, 'Database open read-only' - - self.fireAuditors('retire', nodeid, None) - - node = self.db.getnode(self.classname, nodeid) - node[self.db.RETIRED_FLAG] = 1 - self.db.setnode(self.classname, nodeid, node) - if self.do_journal: - self.db.addjournal(self.classname, nodeid, 'retired', None) - - self.fireReactors('retire', nodeid, None) - - def is_retired(self, nodeid): - '''Return true if the node is retired. - ''' - node = self.db.getnode(cn, nodeid, cldb) - if node.has_key(self.db.RETIRED_FLAG): - return 1 - return 0 - - def destroy(self, nodeid): - '''Destroy a node. - - WARNING: this method should never be used except in extremely rare - situations where there could never be links to the node being - deleted - WARNING: use retire() instead - WARNING: the properties of this node will not be available ever again - WARNING: really, use retire() instead - - Well, I think that's enough warnings. This method exists mostly to - support the session storage of the cgi interface. - ''' - if self.db.journaltag is None: - raise DatabaseError, 'Database open read-only' - self.db.destroynode(self.classname, nodeid) - - def history(self, nodeid): - '''Retrieve the journal of edits on a particular node. - - 'nodeid' must be the id of an existing node of this class or an - IndexError is raised. - - The returned list contains tuples of the form - - (date, tag, action, params) - - 'date' is a Timestamp object specifying the time of the change and - 'tag' is the journaltag specified when the database was opened. - ''' - if not self.do_journal: - raise ValueError, 'Journalling is disabled for this class' - return self.db.getjournal(self.classname, nodeid) - - # Locating nodes: - def hasnode(self, nodeid): - '''Determine if the given nodeid actually exists - ''' - return self.db.hasnode(self.classname, nodeid) - - def setkey(self, propname): - '''Select a String property of this class to be the key property. - - 'propname' must be the name of a String property of this class or - None, or a TypeError is raised. The values of the key property on - all existing nodes must be unique or a ValueError is raised. If the - property doesn't exist, KeyError is raised. - ''' - prop = self.getprops()[propname] - if not isinstance(prop, String): - raise TypeError, 'key properties must be String' - self.key = propname - - def getkey(self): - '''Return the name of the key property for this class or None.''' - return self.key - - def labelprop(self, default_to_id=0): - ''' Return the property name for a label for the given node. - - This method attempts to generate a consistent label for the node. - It tries the following in order: - 1. key property - 2. "name" property - 3. "title" property - 4. first property from the sorted property name list - ''' - k = self.getkey() - if k: - return k - props = self.getprops() - if props.has_key('name'): - return 'name' - elif props.has_key('title'): - return 'title' - if default_to_id: - return 'id' - props = props.keys() - props.sort() - return props[0] - - # TODO: set up a separate index db file for this? profile? - def lookup(self, keyvalue): - '''Locate a particular node by its key property and return its id. - - If this class has no key property, a TypeError is raised. If the - 'keyvalue' matches one of the values for the key property among - the nodes in this class, the matching node's id is returned; - otherwise a KeyError is raised. - ''' - if not self.key: - raise TypeError, 'No key property set for class %s'%self.classname - cldb = self.db.getclassdb(self.classname) - try: - for nodeid in self.db.getnodeids(self.classname, cldb): - node = self.db.getnode(self.classname, nodeid, cldb) - if node.has_key(self.db.RETIRED_FLAG): - continue - if node[self.key] == keyvalue: - cldb.close() - return nodeid - finally: - cldb.close() - raise KeyError, 'No key (%s) value "%s" for "%s"'%(self.key, - keyvalue, self.classname) - - # change from spec - allows multiple props to match - def find(self, **propspec): - '''Get the ids of nodes in this class which link to the given nodes. - - 'propspec' consists of keyword args propname=nodeid or - propname={nodeid:1, } - 'propname' must be the name of a property in this class, or a - KeyError is raised. That property must be a Link or - Multilink property, or a TypeError is raised. - - Any node in this class whose 'propname' property links to any of the - nodeids will be returned. Used by the full text indexing, which knows - that "foo" occurs in msg1, msg3 and file7, so we have hits on these - issues: - - db.issue.find(messages={'1':1,'3':1}, files={'7':1}) - ''' - propspec = propspec.items() - for propname, nodeids in propspec: - # check the prop is OK - prop = self.properties[propname] - if not isinstance(prop, Link) and not isinstance(prop, Multilink): - raise TypeError, "'%s' not a Link/Multilink property"%propname - - # ok, now do the find - cldb = self.db.getclassdb(self.classname) - l = [] - try: - for id in self.db.getnodeids(self.classname, db=cldb): - node = self.db.getnode(self.classname, id, db=cldb) - if node.has_key(self.db.RETIRED_FLAG): - continue - for propname, nodeids in propspec: - # can't test if the node doesn't have this property - if not node.has_key(propname): - continue - if type(nodeids) is type(''): - nodeids = {nodeids:1} - prop = self.properties[propname] - value = node[propname] - if isinstance(prop, Link) and nodeids.has_key(value): - l.append(id) - break - elif isinstance(prop, Multilink): - hit = 0 - for v in value: - if nodeids.has_key(v): - l.append(id) - hit = 1 - break - if hit: - break - finally: - cldb.close() - return l - - def stringFind(self, **requirements): - '''Locate a particular node by matching a set of its String - properties in a caseless search. - - If the property is not a String property, a TypeError is raised. - - The return is a list of the id of all nodes that match. - ''' - for propname in requirements.keys(): - prop = self.properties[propname] - if isinstance(not prop, String): - raise TypeError, "'%s' not a String property"%propname - requirements[propname] = requirements[propname].lower() - l = [] - cldb = self.db.getclassdb(self.classname) - try: - for nodeid in self.db.getnodeids(self.classname, cldb): - node = self.db.getnode(self.classname, nodeid, cldb) - if node.has_key(self.db.RETIRED_FLAG): - continue - for key, value in requirements.items(): - if node[key] is None or node[key].lower() != value: - break - else: - l.append(nodeid) - finally: - cldb.close() - return l - - def list(self): - ''' Return a list of the ids of the active nodes in this class. - ''' - l = [] - cn = self.classname - cldb = self.db.getclassdb(cn) - try: - for nodeid in self.db.getnodeids(cn, cldb): - node = self.db.getnode(cn, nodeid, cldb) - if node.has_key(self.db.RETIRED_FLAG): - continue - l.append(nodeid) - finally: - cldb.close() - l.sort() - return l - - def filter(self, search_matches, filterspec, sort, group, - num_re = re.compile('^\d+$')): - ''' Return a list of the ids of the active nodes in this class that - match the 'filter' spec, sorted by the group spec and then the - sort spec. - - "filterspec" is {propname: value(s)} - "sort" and "group" are (dir, prop) where dir is '+', '-' or None - and prop is a prop name or None - "search_matches" is {nodeid: marker} - - The filter must match all properties specificed - but if the - property value to match is a list, any one of the values in the - list may match for that property to match. - ''' - cn = self.classname - - # optimise filterspec - l = [] - props = self.getprops() - LINK = 0 - MULTILINK = 1 - STRING = 2 - OTHER = 6 - for k, v in filterspec.items(): - propclass = props[k] - if isinstance(propclass, Link): - if type(v) is not type([]): - v = [v] - # replace key values with node ids - u = [] - link_class = self.db.classes[propclass.classname] - for entry in v: - if entry == '-1': entry = None - elif not num_re.match(entry): - try: - entry = link_class.lookup(entry) - except (TypeError,KeyError): - raise ValueError, 'property "%s": %s not a %s'%( - k, entry, self.properties[k].classname) - u.append(entry) - - l.append((LINK, k, u)) - elif isinstance(propclass, Multilink): - if type(v) is not type([]): - v = [v] - # replace key values with node ids - u = [] - link_class = self.db.classes[propclass.classname] - for entry in v: - if not num_re.match(entry): - try: - entry = link_class.lookup(entry) - except (TypeError,KeyError): - raise ValueError, 'new property "%s": %s not a %s'%( - k, entry, self.properties[k].classname) - u.append(entry) - l.append((MULTILINK, k, u)) - elif isinstance(propclass, String): - # simple glob searching - v = re.sub(r'([\|\{\}\\\.\+\[\]\(\)])', r'\\\1', v) - v = v.replace('?', '.') - v = v.replace('*', '.*?') - l.append((STRING, k, re.compile(v, re.I))) - elif isinstance(propclass, Boolean): - if type(v) is type(''): - bv = v.lower() in ('yes', 'true', 'on', '1') - else: - bv = v - l.append((OTHER, k, bv)) - elif isinstance(propclass, Number): - l.append((OTHER, k, int(v))) - else: - l.append((OTHER, k, v)) - filterspec = l - - # now, find all the nodes that are active and pass filtering - l = [] - cldb = self.db.getclassdb(cn) - try: - # TODO: only full-scan once (use items()) - for nodeid in self.db.getnodeids(cn, cldb): - node = self.db.getnode(cn, nodeid, cldb) - if node.has_key(self.db.RETIRED_FLAG): - continue - # apply filter - for t, k, v in filterspec: - # make sure the node has the property - if not node.has_key(k): - # this node doesn't have this property, so reject it - break - - # now apply the property filter - if t == LINK: - # link - if this node's property doesn't appear in the - # filterspec's nodeid list, skip it - if node[k] not in v: - break - elif t == MULTILINK: - # multilink - if any of the nodeids required by the - # filterspec aren't in this node's property, then skip - # it - have = node[k] - for want in v: - if want not in have: - break - else: - continue - break - elif t == STRING: - # RE search - if node[k] is None or not v.search(node[k]): - break - elif t == OTHER: - # straight value comparison for the other types - if node[k] != v: - break - else: - l.append((nodeid, node)) - finally: - cldb.close() - l.sort() - - # filter based on full text search - if search_matches is not None: - k = [] - for v in l: - if search_matches.has_key(v[0]): - k.append(v) - l = k - - # now, sort the result - def sortfun(a, b, sort=sort, group=group, properties=self.getprops(), - db = self.db, cl=self): - a_id, an = a - b_id, bn = b - # sort by group and then sort - for dir, prop in group, sort: - if dir is None or prop is None: continue - - # sorting is class-specific - propclass = properties[prop] - - # handle the properties that might be "faked" - # also, handle possible missing properties - try: - if not an.has_key(prop): - an[prop] = cl.get(a_id, prop) - av = an[prop] - except KeyError: - # the node doesn't have a value for this property - if isinstance(propclass, Multilink): av = [] - else: av = '' - try: - if not bn.has_key(prop): - bn[prop] = cl.get(b_id, prop) - bv = bn[prop] - except KeyError: - # the node doesn't have a value for this property - if isinstance(propclass, Multilink): bv = [] - else: bv = '' - - # String and Date values are sorted in the natural way - if isinstance(propclass, String): - # clean up the strings - if av and av[0] in string.uppercase: - av = an[prop] = av.lower() - if bv and bv[0] in string.uppercase: - bv = bn[prop] = bv.lower() - if (isinstance(propclass, String) or - isinstance(propclass, Date)): - # it might be a string that's really an integer - try: - av = int(av) - bv = int(bv) - except: - pass - if dir == '+': - r = cmp(av, bv) - if r != 0: return r - elif dir == '-': - r = cmp(bv, av) - if r != 0: return r - - # Link properties are sorted according to the value of - # the "order" property on the linked nodes if it is - # present; or otherwise on the key string of the linked - # nodes; or finally on the node ids. - elif isinstance(propclass, Link): - link = db.classes[propclass.classname] - if av is None and bv is not None: return -1 - if av is not None and bv is None: return 1 - if av is None and bv is None: continue - if link.getprops().has_key('order'): - if dir == '+': - r = cmp(link.get(av, 'order'), - link.get(bv, 'order')) - if r != 0: return r - elif dir == '-': - r = cmp(link.get(bv, 'order'), - link.get(av, 'order')) - if r != 0: return r - elif link.getkey(): - key = link.getkey() - if dir == '+': - r = cmp(link.get(av, key), link.get(bv, key)) - if r != 0: return r - elif dir == '-': - r = cmp(link.get(bv, key), link.get(av, key)) - if r != 0: return r - else: - if dir == '+': - r = cmp(av, bv) - if r != 0: return r - elif dir == '-': - r = cmp(bv, av) - if r != 0: return r - - # Multilink properties are sorted according to how many - # links are present. - elif isinstance(propclass, Multilink): - if dir == '+': - r = cmp(len(av), len(bv)) - if r != 0: return r - elif dir == '-': - r = cmp(len(bv), len(av)) - if r != 0: return r - elif isinstance(propclass, Number) or isinstance(propclass, Boolean): - if dir == '+': - r = cmp(av, bv) - elif dir == '-': - r = cmp(bv, av) - - # end for dir, prop in sort, group: - # if all else fails, compare the ids - return cmp(a[0], b[0]) - - l.sort(sortfun) - return [i[0] for i in l] - - def count(self): - '''Get the number of nodes in this class. - - If the returned integer is 'numnodes', the ids of all the nodes - in this class run from 1 to numnodes, and numnodes+1 will be the - id of the next node to be created in this class. - ''' - return self.db.countnodes(self.classname) - - # Manipulating properties: - - def getprops(self, protected=1): - '''Return a dictionary mapping property names to property objects. - If the "protected" flag is true, we include protected properties - - those which may not be modified. - - In addition to the actual properties on the node, these - methods provide the "creation" and "activity" properties. If the - "protected" flag is true, we include protected properties - those - which may not be modified. - ''' - d = self.properties.copy() - if protected: - d['id'] = String() - d['creation'] = hyperdb.Date() - d['activity'] = hyperdb.Date() - d['creator'] = hyperdb.Link('user') - return d - - def addprop(self, **properties): - '''Add properties to this class. - - The keyword arguments in 'properties' must map names to property - objects, or a TypeError is raised. None of the keys in 'properties' - may collide with the names of existing properties, or a ValueError - is raised before any properties have been added. - ''' - for key in properties.keys(): - if self.properties.has_key(key): - raise ValueError, key - self.properties.update(properties) - - def index(self, nodeid): - '''Add (or refresh) the node to search indexes - ''' - # find all the String properties that have indexme - for prop, propclass in self.getprops().items(): - if isinstance(propclass, String) and propclass.indexme: - try: - value = str(self.get(nodeid, prop)) - except IndexError: - # node no longer exists - entry should be removed - self.db.indexer.purge_entry((self.classname, nodeid, prop)) - else: - # and index them under (classname, nodeid, property) - self.db.indexer.add_text((self.classname, nodeid, prop), - value) - - # - # Detector interface - # - def audit(self, event, detector): - '''Register a detector - ''' - l = self.auditors[event] - if detector not in l: - self.auditors[event].append(detector) - - def fireAuditors(self, action, nodeid, newvalues): - '''Fire all registered auditors. - ''' - for audit in self.auditors[action]: - audit(self.db, self, nodeid, newvalues) - - def react(self, event, detector): - '''Register a detector - ''' - l = self.reactors[event] - if detector not in l: - self.reactors[event].append(detector) - - def fireReactors(self, action, nodeid, oldvalues): - '''Fire all registered reactors. - ''' - for react in self.reactors[action]: - react(self.db, self, nodeid, oldvalues) - -class FileClass(Class): - '''This class defines a large chunk of data. To support this, it has a - mandatory String property "content" which is typically saved off - externally to the hyperdb. - - The default MIME type of this data is defined by the - "default_mime_type" class attribute, which may be overridden by each - node if the class defines a "type" String property. - ''' - default_mime_type = 'text/plain' - - def create(self, **propvalues): - ''' snaffle the file propvalue and store in a file - ''' - content = propvalues['content'] - del propvalues['content'] - newid = Class.create(self, **propvalues) - self.db.storefile(self.classname, newid, None, content) - return newid - - def import_list(self, propnames, proplist): - ''' Trap the "content" property... - ''' - # dupe this list so we don't affect others - propnames = propnames[:] - - # extract the "content" property from the proplist - i = propnames.index('content') - content = eval(proplist[i]) - del propnames[i] - del proplist[i] - - # do the normal import - newid = Class.import_list(self, propnames, proplist) - - # save off the "content" file - self.db.storefile(self.classname, newid, None, content) - return newid - - def get(self, nodeid, propname, default=_marker, cache=1): - ''' trap the content propname and get it from the file - ''' - - poss_msg = 'Possibly a access right configuration problem.' - if propname == 'content': - try: - return self.db.getfile(self.classname, nodeid, None) - except IOError, (strerror): - # BUG: by catching this we donot see an error in the log. - return 'ERROR reading file: %s%s\n%s\n%s'%( - self.classname, nodeid, poss_msg, strerror) - if default is not _marker: - return Class.get(self, nodeid, propname, default, cache=cache) - else: - return Class.get(self, nodeid, propname, cache=cache) - - def getprops(self, protected=1): - ''' In addition to the actual properties on the node, these methods - provide the "content" property. If the "protected" flag is true, - we include protected properties - those which may not be - modified. - ''' - d = Class.getprops(self, protected=protected).copy() - d['content'] = hyperdb.String() - return d - - def index(self, nodeid): - ''' Index the node in the search index. - - We want to index the content in addition to the normal String - property indexing. - ''' - # perform normal indexing - Class.index(self, nodeid) - - # get the content to index - content = self.get(nodeid, 'content') - - # figure the mime type - if self.properties.has_key('type'): - mime_type = self.get(nodeid, 'type') - else: - mime_type = self.default_mime_type - - # and index! - self.db.indexer.add_text((self.classname, nodeid, 'content'), content, - mime_type) - -# deviation from spec - was called ItemClass -class IssueClass(Class, roundupdb.IssueClass): - # Overridden methods: - def __init__(self, db, classname, **properties): - '''The newly-created class automatically includes the "messages", - "files", "nosy", and "superseder" properties. If the 'properties' - dictionary attempts to specify any of these properties or a - "creation" or "activity" property, a ValueError is raised. - ''' - if not properties.has_key('title'): - properties['title'] = hyperdb.String(indexme='yes') - if not properties.has_key('messages'): - properties['messages'] = hyperdb.Multilink("msg") - if not properties.has_key('files'): - properties['files'] = hyperdb.Multilink("file") - if not properties.has_key('nosy'): - # note: journalling is turned off as it really just wastes - # space. this behaviour may be overridden in an instance - properties['nosy'] = hyperdb.Multilink("user", do_journal="no") - if not properties.has_key('superseder'): - properties['superseder'] = hyperdb.Multilink(classname) - Class.__init__(self, db, classname, **properties) - -#
--- a/roundup/backends/back_bsddb3.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,105 +0,0 @@ -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -#$Id: back_bsddb3.py,v 1.17 2002-09-13 08:20:12 richard Exp $ - -import bsddb3, os, marshal -from roundup import hyperdb, date - -# these classes are so similar, we just use the anydbm methods -from back_anydbm import Database, Class, FileClass, IssueClass - -# -# Now the database -# -class Database(Database): - """A database for storing records containing flexible data types.""" - # - # Class DBs - # - def clear(self): - for cn in self.classes.keys(): - db = os.path.join(self.dir, 'nodes.%s'%cn) - bsddb3.btopen(db, 'n') - db = os.path.join(self.dir, 'journals.%s'%cn) - bsddb3.btopen(db, 'n') - - def getclassdb(self, classname, mode='r'): - ''' grab a connection to the class db that will be used for - multiple actions - ''' - path = os.path.join(os.getcwd(), self.dir, 'nodes.%s'%classname) - if os.path.exists(path): - return bsddb3.btopen(path, mode) - else: - return bsddb3.btopen(path, 'c') - - def opendb(self, name, mode): - '''Low-level database opener that gets around anydbm/dbm - eccentricities. - ''' - if __debug__: - print >>hyperdb.DEBUG, self, 'opendb', (self, name, mode) - # determine which DB wrote the class file - path = os.path.join(os.getcwd(), self.dir, name) - if not os.path.exists(path): - if __debug__: - print >>hyperdb.DEBUG, "opendb bsddb3.open(%r, 'c')"%path - return bsddb3.btopen(path, 'c') - - # open the database with the correct module - if __debug__: - print >>hyperdb.DEBUG, "opendb bsddb3.open(%r, %r)"%(path, mode) - return bsddb3.btopen(path, mode) - - # - # Journal - # - def getjournal(self, classname, nodeid): - ''' get the journal for id - ''' - # attempt to open the journal - in some rare cases, the journal may - # not exist - try: - db = bsddb3.btopen(os.path.join(self.dir, 'journals.%s'%classname), - 'r') - except bsddb3._db.DBNoSuchFileError: - raise IndexError, 'no such %s %s'%(classname, nodeid) - # more handling of bad journals - if not db.has_key(nodeid): - raise IndexError, 'no such %s %s'%(classname, nodeid) - journal = marshal.loads(db[nodeid]) - res = [] - for entry in journal: - (nodeid, date_stamp, user, action, params) = entry - date_obj = date.Date(date_stamp) - res.append((nodeid, date_obj, user, action, params)) - db.close() - return res - - def getCachedJournalDB(self, classname): - ''' get the journal db, looking in our cache of databases for commit - ''' - # get the database handle - db_name = 'journals.%s'%classname - if self.databases.has_key(db_name): - return self.databases[db_name] - else: - db = bsddb3.btopen(os.path.join(self.dir, db_name), 'c') - self.databases[db_name] = db - return db -
--- a/roundup/backends/back_gadfly.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,264 +0,0 @@ -# $Id: back_gadfly.py,v 1.27 2002-09-26 03:04:24 richard Exp $ -__doc__ = ''' -About Gadfly -============ - -Gadfly is a collection of python modules that provides relational -database functionality entirely implemented in Python. It supports a -subset of the intergalactic standard RDBMS Structured Query Language -SQL. - - -Basic Structure -=============== - -We map roundup classes to relational tables. Automatically detect schema -changes and modify the gadfly table schemas appropriately. Multilinks -(which represent a many-to-many relationship) are handled through -intermediate tables. - -Journals are stored adjunct to the per-class tables. - -Table names and columns have "_" prepended so the names can't -clash with restricted names (like "order"). Retirement is determined by the -__retired__ column being true. - -All columns are defined as VARCHAR, since it really doesn't matter what -type they're defined as. We stuff all kinds of data in there ;) [as long as -it's marshallable, gadfly doesn't care] - - -Additional Instance Requirements -================================ - -The instance configuration must specify where the database is. It does this -with GADFLY_DATABASE, which is used as the arguments to the gadfly.gadfly() -method: - -Using an on-disk database directly (not a good idea): - GADFLY_DATABASE = (database name, directory) - -Using a network database (much better idea): - GADFLY_DATABASE = (policy, password, address, port) - -Because multiple accesses directly to a gadfly database aren't handled, but -multiple network accesses are, it's strongly advised that the latter setup be -used. - -''' - -# standard python modules -import sys, os, time, re, errno, weakref, copy - -# roundup modules -from roundup import hyperdb, date, password, roundupdb, security -from roundup.hyperdb import String, Password, Date, Interval, Link, \ - Multilink, DatabaseError, Boolean, Number - -# basic RDBMS backen implementation -from roundup.backends import rdbms_common - -# the all-important gadfly :) -import gadfly -import gadfly.client -import gadfly.database - -class Database(rdbms_common.Database): - # char to use for positional arguments - arg = '?' - - def open_connection(self): - db = getattr(self.config, 'GADFLY_DATABASE', ('database', self.dir)) - if len(db) == 2: - # ensure files are group readable and writable - os.umask(0002) - try: - self.conn = gadfly.gadfly(*db) - except IOError, error: - if error.errno != errno.ENOENT: - raise - self.database_schema = {} - self.conn = gadfly.gadfly() - self.conn.startup(*db) - self.cursor = self.conn.cursor() - self.cursor.execute('create table schema (schema varchar)') - self.cursor.execute('create table ids (name varchar, num integer)') - else: - self.cursor = self.conn.cursor() - self.cursor.execute('select schema from schema') - self.database_schema = self.cursor.fetchone()[0] - else: - self.conn = gadfly.client.gfclient(*db) - self.database_schema = self.load_dbschema() - - def __repr__(self): - return '<roundfly 0x%x>'%id(self) - - def sql_fetchone(self): - ''' Fetch a single row. If there's nothing to fetch, return None. - ''' - try: - return self.cursor.fetchone() - except gadfly.database.error, message: - if message == 'no more results': - return None - raise - - def sql_fetchall(self): - ''' Fetch a single row. If there's nothing to fetch, return []. - ''' - try: - return self.cursor.fetchall() - except gadfly.database.error, message: - if message == 'no more results': - return [] - raise - - def save_dbschema(self, schema): - ''' Save the schema definition that the database currently implements - ''' - self.sql('insert into schema values (?)', (self.database_schema,)) - - def load_dbschema(self): - ''' Load the schema definition that the database currently implements - ''' - self.cursor.execute('select schema from schema') - return self.cursor.fetchone()[0] - - def save_journal(self, classname, cols, nodeid, journaldate, - journaltag, action, params): - ''' Save the journal entry to the database - ''' - # nothing special to do - entry = (nodeid, journaldate, journaltag, action, params) - - # do the insert - a = self.arg - sql = 'insert into %s__journal (%s) values (?,?,?,?,?)'%(classname, - cols) - if __debug__: - print >>hyperdb.DEBUG, 'addjournal', (self, sql, entry) - self.cursor.execute(sql, entry) - - def load_journal(self, classname, cols, nodeid): - ''' Load the journal from the database - ''' - # now get the journal entries - sql = 'select %s from %s__journal where nodeid=%s'%(cols, classname, - self.arg) - if __debug__: - print >>hyperdb.DEBUG, 'getjournal', (self, sql, nodeid) - self.cursor.execute(sql, (nodeid,)) - res = [] - for nodeid, date_stamp, user, action, params in self.cursor.fetchall(): - res.append((nodeid, date.Date(date_stamp), user, action, params)) - return res - -class GadflyClass: - def filter(self, search_matches, filterspec, sort, group): - ''' Gadfly doesn't have a LIKE predicate :( - ''' - cn = self.classname - - # figure the WHERE clause from the filterspec - props = self.getprops() - frum = ['_'+cn] - where = [] - args = [] - a = self.db.arg - for k, v in filterspec.items(): - propclass = props[k] - if isinstance(propclass, Multilink): - tn = '%s_%s'%(cn, k) - frum.append(tn) - if isinstance(v, type([])): - s = ','.join([a for x in v]) - where.append('id=%s.nodeid and %s.linkid in (%s)'%(tn,tn,s)) - args = args + v - else: - where.append('id=%s.nodeid and %s.linkid = %s'%(tn, tn, a)) - args.append(v) - else: - if isinstance(v, type([])): - s = ','.join([a for x in v]) - where.append('_%s in (%s)'%(k, s)) - args = args + v - else: - where.append('_%s=%s'%(k, a)) - args.append(v) - - # add results of full text search - if search_matches is not None: - v = search_matches.keys() - s = ','.join([a for x in v]) - where.append('id in (%s)'%s) - args = args + v - - # "grouping" is just the first-order sorting in the SQL fetch - # can modify it...) - orderby = [] - ordercols = [] - if group[0] is not None and group[1] is not None: - if group[0] != '-': - orderby.append('_'+group[1]) - ordercols.append('_'+group[1]) - else: - orderby.append('_'+group[1]+' desc') - ordercols.append('_'+group[1]) - - # now add in the sorting - group = '' - if sort[0] is not None and sort[1] is not None: - direction, colname = sort - if direction != '-': - if colname == 'id': - orderby.append(colname) - else: - orderby.append('_'+colname) - ordercols.append('_'+colname) - else: - if colname == 'id': - orderby.append(colname+' desc') - ordercols.append(colname) - else: - orderby.append('_'+colname+' desc') - ordercols.append('_'+colname) - - # construct the SQL - frum = ','.join(frum) - if where: - where = ' where ' + (' and '.join(where)) - else: - where = '' - cols = ['id'] - if orderby: - cols = cols + ordercols - order = ' order by %s'%(','.join(orderby)) - else: - order = '' - cols = ','.join(cols) - sql = 'select %s from %s %s%s%s'%(cols, frum, where, group, order) - args = tuple(args) - if __debug__: - print >>hyperdb.DEBUG, 'filter', (self, sql, args) - self.db.cursor.execute(sql, args) - l = self.db.cursor.fetchall() - - # return the IDs - return [row[0] for row in l] - - def find(self, **propspec): - ''' Overload to filter out duplicates in the result - ''' - d = {} - for k in rdbms_common.Class.find(self, **propspec): - d[k] = 1 - return d.keys() - -class Class(GadflyClass, rdbms_common.Class): - pass -class IssueClass(GadflyClass, rdbms_common.IssueClass): - pass -class FileClass(GadflyClass, rdbms_common.FileClass): - pass -
--- a/roundup/backends/back_metakit.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1278 +0,0 @@ -from roundup import hyperdb, date, password, roundupdb, security -import metakit -from sessions import Sessions -import re, marshal, os, sys, weakref, time, calendar -from roundup import indexer -import locking - -_dbs = {} - -def Database(config, journaltag=None): - db = _dbs.get(config.DATABASE, None) - if db is None or db._db is None: - db = _Database(config, journaltag) - _dbs[config.DATABASE] = db - else: - db.journaltag = journaltag - try: - delattr(db, 'curuserid') - except AttributeError: - pass - return db - -class _Database(hyperdb.Database): - def __init__(self, config, journaltag=None): - self.config = config - self.journaltag = journaltag - self.classes = {} - self.dirty = 0 - self.lockfile = None - self._db = self.__open() - self.indexer = Indexer(self.config.DATABASE, self._db) - self.sessions = Sessions(self.config) - self.security = security.Security(self) - - os.umask(0002) - - def post_init(self): - if self.indexer.should_reindex(): - self.reindex() - - def reindex(self): - for klass in self.classes.values(): - for nodeid in klass.list(): - klass.index(nodeid) - self.indexer.save_index() - - - # --- defined in ping's spec - def __getattr__(self, classname): - if classname == 'curuserid': - if self.journaltag is None: - return None - - try: - self.curuserid = x = int(self.classes['user'].lookup(self.journaltag)) - except KeyError: - if self.journaltag == 'admin': - self.curuserid = x = 1 - else: - x = 0 - return x - elif classname == 'transactions': - return self.dirty - return self.getclass(classname) - def getclass(self, classname): - try: - return self.classes[classname] - except KeyError: - raise KeyError, 'There is no class called "%s"'%classname - def getclasses(self): - return self.classes.keys() - # --- end of ping's spec - # --- exposed methods - def commit(self): - if self.dirty: - self._db.commit() - for cl in self.classes.values(): - cl._commit() - self.indexer.save_index() - self.dirty = 0 - def rollback(self): - if self.dirty: - for cl in self.classes.values(): - cl._rollback() - self._db.rollback() - self._db = None - self._db = metakit.storage(self.dbnm, 1) - self.hist = self._db.view('history') - self.tables = self._db.view('tables') - self.indexer.datadb = self._db - self.dirty = 0 - def clearCache(self): - for cl in self.classes.values(): - cl._commit() - def clear(self): - for cl in self.classes.values(): - cl._clear() - def hasnode(self, classname, nodeid): - return self.getclass(classname).hasnode(nodeid) - def pack(self, pack_before): - mindate = int(calendar.timegm(pack_before.get_tuple())) - i = 0 - while i < len(self.hist): - if self.hist[i].date < mindate and self.hist[i].action != _CREATE: - self.hist.delete(i) - else: - i = i + 1 - def addclass(self, cl): - self.classes[cl.classname] = cl - if self.tables.find(name=cl.classname) < 0: - self.tables.append(name=cl.classname) - def addjournal(self, tablenm, nodeid, action, params, creator=None, - creation=None): - tblid = self.tables.find(name=tablenm) - if tblid == -1: - tblid = self.tables.append(name=tablenm) - if creator is None: - creator = self.curuserid - else: - try: - creator = int(creator) - except TypeError: - creator = int(self.getclass('user').lookup(creator)) - if creation is None: - creation = int(time.time()) - elif isinstance(creation, date.Date): - creation = int(calendar.timegm(creation.get_tuple())) - # tableid:I,nodeid:I,date:I,user:I,action:I,params:B - self.hist.append(tableid=tblid, - nodeid=int(nodeid), - date=creation, - action=action, - user = creator, - params = marshal.dumps(params)) - def getjournal(self, tablenm, nodeid): - rslt = [] - tblid = self.tables.find(name=tablenm) - if tblid == -1: - return rslt - q = self.hist.select(tableid=tblid, nodeid=int(nodeid)) - if len(q) == 0: - raise IndexError, "no history for id %s in %s" % (nodeid, tablenm) - i = 0 - #userclass = self.getclass('user') - for row in q: - try: - params = marshal.loads(row.params) - except ValueError: - print "history couldn't unmarshal %r" % row.params - params = {} - #usernm = userclass.get(str(row.user), 'username') - dt = date.Date(time.gmtime(row.date)) - #rslt.append((nodeid, dt, usernm, _actionnames[row.action], params)) - rslt.append((nodeid, dt, str(row.user), _actionnames[row.action], params)) - return rslt - - def destroyjournal(self, tablenm, nodeid): - nodeid = int(nodeid) - tblid = self.tables.find(name=tablenm) - if tblid == -1: - return - i = 0 - hist = self.hist - while i < len(hist): - if hist[i].tableid == tblid and hist[i].nodeid == nodeid: - hist.delete(i) - else: - i = i + 1 - self.dirty = 1 - - def close(self): - for cl in self.classes.values(): - cl.db = None - self._db = None - if self.lockfile is not None: - locking.release_lock(self.lockfile) - if _dbs.has_key(self.config.DATABASE): - del _dbs[self.config.DATABASE] - if self.lockfile is not None: - self.lockfile.close() - self.lockfile = None - self.classes = {} - self.indexer = None - - # --- internal - def __open(self): - if not os.path.exists(self.config.DATABASE): - os.makedirs(self.config.DATABASE) - self.dbnm = db = os.path.join(self.config.DATABASE, 'tracker.mk4') - lockfilenm = db[:-3]+'lck' - self.lockfile = locking.acquire_lock(lockfilenm) - self.lockfile.write(str(os.getpid())) - self.lockfile.flush() - self.fastopen = 0 - if os.path.exists(db): - dbtm = os.path.getmtime(db) - pkgnm = self.config.__name__.split('.')[0] - schemamod = sys.modules.get(pkgnm+'.dbinit', None) - if schemamod: - if os.path.exists(schemamod.__file__): - schematm = os.path.getmtime(schemamod.__file__) - if schematm < dbtm: - # found schema mod - it's older than the db - self.fastopen = 1 - else: - # can't find schemamod - must be frozen - self.fastopen = 1 - db = metakit.storage(db, 1) - hist = db.view('history') - tables = db.view('tables') - if not self.fastopen: - if not hist.structure(): - hist = db.getas('history[tableid:I,nodeid:I,date:I,user:I,action:I,params:B]') - if not tables.structure(): - tables = db.getas('tables[name:S]') - db.commit() - self.tables = tables - self.hist = hist - return db - -_STRINGTYPE = type('') -_LISTTYPE = type([]) -_CREATE, _SET, _RETIRE, _LINK, _UNLINK = range(5) - -_actionnames = { - _CREATE : 'create', - _SET : 'set', - _RETIRE : 'retire', - _LINK : 'link', - _UNLINK : 'unlink', -} - -_marker = [] - -_ALLOWSETTINGPRIVATEPROPS = 0 - -class Class: - privateprops = None - def __init__(self, db, classname, **properties): - #self.db = weakref.proxy(db) - self.db = db - self.classname = classname - self.keyname = None - self.ruprops = properties - self.privateprops = { 'id' : hyperdb.String(), - 'activity' : hyperdb.Date(), - 'creation' : hyperdb.Date(), - 'creator' : hyperdb.Link('user') } - self.auditors = {'create': [], 'set': [], 'retire': []} # event -> list of callables - self.reactors = {'create': [], 'set': [], 'retire': []} # ditto - view = self.__getview() - self.maxid = 1 - if view: - self.maxid = view[-1].id + 1 - self.uncommitted = {} - self.rbactions = [] - # people reach inside!! - self.properties = self.ruprops - self.db.addclass(self) - self.idcache = {} - - # default is to journal changes - self.do_journal = 1 - - def enableJournalling(self): - '''Turn journalling on for this class - ''' - self.do_journal = 1 - - def disableJournalling(self): - '''Turn journalling off for this class - ''' - self.do_journal = 0 - - # --- the roundup.Class methods - def audit(self, event, detector): - l = self.auditors[event] - if detector not in l: - self.auditors[event].append(detector) - def fireAuditors(self, action, nodeid, newvalues): - for audit in self.auditors[action]: - audit(self.db, self, nodeid, newvalues) - def fireReactors(self, action, nodeid, oldvalues): - for react in self.reactors[action]: - react(self.db, self, nodeid, oldvalues) - def react(self, event, detector): - l = self.reactors[event] - if detector not in l: - self.reactors[event].append(detector) - # --- the hyperdb.Class methods - def create(self, **propvalues): - self.fireAuditors('create', None, propvalues) - rowdict = {} - rowdict['id'] = newid = self.maxid - self.maxid += 1 - ndx = self.getview(1).append(rowdict) - propvalues['#ISNEW'] = 1 - try: - self.set(str(newid), **propvalues) - except Exception: - self.maxid -= 1 - raise - return str(newid) - - def get(self, nodeid, propname, default=_marker, cache=1): - # default and cache aren't in the spec - # cache=0 means "original value" - - view = self.getview() - id = int(nodeid) - if cache == 0: - oldnode = self.uncommitted.get(id, None) - if oldnode and oldnode.has_key(propname): - return oldnode[propname] - ndx = self.idcache.get(id, None) - if ndx is None: - ndx = view.find(id=id) - if ndx < 0: - raise IndexError, "%s has no node %s" % (self.classname, nodeid) - self.idcache[id] = ndx - try: - raw = getattr(view[ndx], propname) - except AttributeError: - raise KeyError, propname - rutyp = self.ruprops.get(propname, None) - if rutyp is None: - rutyp = self.privateprops[propname] - converter = _converters.get(rutyp.__class__, None) - if converter: - raw = converter(raw) - return raw - - def set(self, nodeid, **propvalues): - isnew = 0 - if propvalues.has_key('#ISNEW'): - isnew = 1 - del propvalues['#ISNEW'] - if not isnew: - self.fireAuditors('set', nodeid, propvalues) - if not propvalues: - return propvalues - if propvalues.has_key('id'): - raise KeyError, '"id" is reserved' - if self.db.journaltag is None: - raise hyperdb.DatabaseError, 'Database open read-only' - view = self.getview(1) - # node must exist & not be retired - id = int(nodeid) - ndx = view.find(id=id) - if ndx < 0: - raise IndexError, "%s has no node %s" % (self.classname, nodeid) - row = view[ndx] - if row._isdel: - raise IndexError, "%s has no node %s" % (self.classname, nodeid) - oldnode = self.uncommitted.setdefault(id, {}) - changes = {} - - for key, value in propvalues.items(): - # this will raise the KeyError if the property isn't valid - # ... we don't use getprops() here because we only care about - # the writeable properties. - if _ALLOWSETTINGPRIVATEPROPS: - prop = self.ruprops.get(key, None) - if not prop: - prop = self.privateprops[key] - else: - prop = self.ruprops[key] - converter = _converters.get(prop.__class__, lambda v: v) - # if the value's the same as the existing value, no sense in - # doing anything - oldvalue = converter(getattr(row, key)) - if value == oldvalue: - del propvalues[key] - continue - - # check to make sure we're not duplicating an existing key - if key == self.keyname: - iv = self.getindexview(1) - ndx = iv.find(k=value) - if ndx == -1: - iv.append(k=value, i=row.id) - if not isnew: - ndx = iv.find(k=oldvalue) - if ndx > -1: - iv.delete(ndx) - else: - raise ValueError, 'node with key "%s" exists'%value - - # do stuff based on the prop type - if isinstance(prop, hyperdb.Link): - link_class = prop.classname - # must be a string or None - if value is not None and not isinstance(value, type('')): - raise ValueError, 'property "%s" link value be a string'%( - key) - # Roundup sets to "unselected" by passing None - if value is None: - value = 0 - # if it isn't a number, it's a key - try: - int(value) - except ValueError: - try: - value = self.db.getclass(link_class).lookup(value) - except (TypeError, KeyError): - raise IndexError, 'new property "%s": %s not a %s'%( - key, value, prop.classname) - - if (value is not None and - not self.db.getclass(link_class).hasnode(value)): - raise IndexError, '%s has no node %s'%(link_class, value) - - setattr(row, key, int(value)) - changes[key] = oldvalue - - if self.do_journal and prop.do_journal: - # register the unlink with the old linked node - if oldvalue: - self.db.addjournal(link_class, value, _UNLINK, - (self.classname, str(row.id), key)) - - # register the link with the newly linked node - if value: - self.db.addjournal(link_class, value, _LINK, - (self.classname, str(row.id), key)) - - elif isinstance(prop, hyperdb.Multilink): - if value is not None and type(value) != _LISTTYPE: - raise TypeError, 'new property "%s" not a list of ids'%key - link_class = prop.classname - l = [] - if value is None: - value = [] - for entry in value: - if type(entry) != _STRINGTYPE: - raise ValueError, 'new property "%s" link value ' \ - 'must be a string'%key - # if it isn't a number, it's a key - try: - int(entry) - except ValueError: - try: - entry = self.db.getclass(link_class).lookup(entry) - except (TypeError, KeyError): - raise IndexError, 'new property "%s": %s not a %s'%( - key, entry, prop.classname) - l.append(entry) - propvalues[key] = value = l - - # handle removals - rmvd = [] - for id in oldvalue: - if id not in value: - rmvd.append(id) - # register the unlink with the old linked node - if self.do_journal and prop.do_journal: - self.db.addjournal(link_class, id, _UNLINK, - (self.classname, str(row.id), key)) - - # handle additions - adds = [] - for id in value: - if id not in oldvalue: - if not self.db.getclass(link_class).hasnode(id): - raise IndexError, '%s has no node %s'%( - link_class, id) - adds.append(id) - # register the link with the newly linked node - if self.do_journal and prop.do_journal: - self.db.addjournal(link_class, id, _LINK, - (self.classname, str(row.id), key)) - - sv = getattr(row, key) - i = 0 - while i < len(sv): - if str(sv[i].fid) in rmvd: - sv.delete(i) - else: - i += 1 - for id in adds: - sv.append(fid=int(id)) - changes[key] = oldvalue - if not rmvd and not adds: - del propvalues[key] - - elif isinstance(prop, hyperdb.String): - if value is not None and type(value) != _STRINGTYPE: - raise TypeError, 'new property "%s" not a string'%key - if value is None: - value = '' - setattr(row, key, value) - changes[key] = oldvalue - if hasattr(prop, 'isfilename') and prop.isfilename: - propvalues[key] = os.path.basename(value) - if prop.indexme and value is not None: - self.db.indexer.add_text((self.classname, nodeid, key), - value, 'text/plain') - - elif isinstance(prop, hyperdb.Password): - if value is not None and not isinstance(value, password.Password): - raise TypeError, 'new property "%s" not a Password'% key - if value is None: - value = '' - setattr(row, key, str(value)) - changes[key] = str(oldvalue) - propvalues[key] = str(value) - - elif isinstance(prop, hyperdb.Date): - if value is not None and not isinstance(value, date.Date): - raise TypeError, 'new property "%s" not a Date'% key - if value is None: - setattr(row, key, 0) - else: - setattr(row, key, int(calendar.timegm(value.get_tuple()))) - changes[key] = str(oldvalue) - propvalues[key] = str(value) - - elif isinstance(prop, hyperdb.Interval): - if value is not None and not isinstance(value, date.Interval): - raise TypeError, 'new property "%s" not an Interval'% key - if value is None: - setattr(row, key, '') - else: - setattr(row, key, str(value)) - changes[key] = str(oldvalue) - propvalues[key] = str(value) - - elif isinstance(prop, hyperdb.Number): - if value is None: - value = 0 - try: - v = int(value) - except ValueError: - raise TypeError, "%s (%s) is not numeric" % (key, repr(value)) - setattr(row, key, v) - changes[key] = oldvalue - propvalues[key] = value - - elif isinstance(prop, hyperdb.Boolean): - if value is None: - bv = 0 - elif value not in (0,1): - raise TypeError, "%s (%s) is not boolean" % (key, repr(value)) - else: - bv = value - setattr(row, key, bv) - changes[key] = oldvalue - propvalues[key] = value - - oldnode[key] = oldvalue - - # nothing to do? - if not propvalues: - return propvalues - if not propvalues.has_key('activity'): - row.activity = int(time.time()) - if isnew: - if not row.creation: - row.creation = int(time.time()) - if not row.creator: - row.creator = self.db.curuserid - - self.db.dirty = 1 - if self.do_journal: - if isnew: - self.db.addjournal(self.classname, nodeid, _CREATE, {}) - self.fireReactors('create', nodeid, None) - else: - self.db.addjournal(self.classname, nodeid, _SET, changes) - self.fireReactors('set', nodeid, oldnode) - - return propvalues - - def retire(self, nodeid): - if self.db.journaltag is None: - raise hyperdb.DatabaseError, 'Database open read-only' - self.fireAuditors('retire', nodeid, None) - view = self.getview(1) - ndx = view.find(id=int(nodeid)) - if ndx < 0: - raise KeyError, "nodeid %s not found" % nodeid - row = view[ndx] - oldvalues = self.uncommitted.setdefault(row.id, {}) - oldval = oldvalues['_isdel'] = row._isdel - row._isdel = 1 - if self.do_journal: - self.db.addjournal(self.classname, nodeid, _RETIRE, {}) - if self.keyname: - iv = self.getindexview(1) - ndx = iv.find(k=getattr(row, self.keyname),i=row.id) - if ndx > -1: - iv.delete(ndx) - self.db.dirty = 1 - self.fireReactors('retire', nodeid, None) - def history(self, nodeid): - if not self.do_journal: - raise ValueError, 'Journalling is disabled for this class' - return self.db.getjournal(self.classname, nodeid) - def setkey(self, propname): - if self.keyname: - if propname == self.keyname: - return - raise ValueError, "%s already indexed on %s" % (self.classname, self.keyname) - prop = self.properties.get(propname, None) - if prop is None: - prop = self.privateprops.get(propname, None) - if prop is None: - raise KeyError, "no property %s" % propname - if not isinstance(prop, hyperdb.String): - raise TypeError, "%s is not a String" % propname - # first setkey for this run - self.keyname = propname - iv = self.db._db.view('_%s' % self.classname) - if self.db.fastopen and iv.structure(): - return - # very first setkey ever - self.db.dirty = 1 - iv = self.db._db.getas('_%s[k:S,i:I]' % self.classname) - iv = iv.ordered(1) -# print "setkey building index" - for row in self.getview(): - iv.append(k=getattr(row, propname), i=row.id) - self.db.commit() - def getkey(self): - return self.keyname - def lookup(self, keyvalue): - if type(keyvalue) is not _STRINGTYPE: - raise TypeError, "%r is not a string" % keyvalue - iv = self.getindexview() - if iv: - ndx = iv.find(k=keyvalue) - if ndx > -1: - return str(iv[ndx].i) - else: - view = self.getview() - ndx = view.find({self.keyname:keyvalue, '_isdel':0}) - if ndx > -1: - return str(view[ndx].id) - raise KeyError, keyvalue - - def destroy(self, id): - view = self.getview(1) - ndx = view.find(id=int(id)) - if ndx > -1: - if self.keyname: - keyvalue = getattr(view[ndx], self.keyname) - iv = self.getindexview(1) - if iv: - ivndx = iv.find(k=keyvalue) - if ivndx > -1: - iv.delete(ivndx) - view.delete(ndx) - self.db.destroyjournal(self.classname, id) - self.db.dirty = 1 - - def find(self, **propspec): - """Get the ids of nodes in this class which link to the given nodes. - - 'propspec' consists of keyword args propname={nodeid:1,} - 'propname' must be the name of a property in this class, or a - KeyError is raised. That property must be a Link or - Multilink property, or a TypeError is raised. - - Any node in this class whose propname property links to any of the - nodeids will be returned. Used by the full text indexing, which knows - that "foo" occurs in msg1, msg3 and file7; so we have hits on these - issues: - - db.issue.find(messages={'1':1,'3':1}, files={'7':1}) - - """ - propspec = propspec.items() - for propname, nodeid in propspec: - # check the prop is OK - prop = self.ruprops[propname] - if (not isinstance(prop, hyperdb.Link) and - not isinstance(prop, hyperdb.Multilink)): - raise TypeError, "'%s' not a Link/Multilink property"%propname - - vws = [] - for propname, ids in propspec: - if type(ids) is _STRINGTYPE: - ids = {ids:1} - prop = self.ruprops[propname] - view = self.getview() - if isinstance(prop, hyperdb.Multilink): - view = view.flatten(getattr(view, propname)) - def ff(row, nm=propname, ids=ids): - return ids.has_key(str(row.fid)) - else: - def ff(row, nm=propname, ids=ids): - return ids.has_key(str(getattr(row, nm))) - ndxview = view.filter(ff) - vws.append(ndxview.unique()) - - # handle the empty match case - if not vws: - return [] - - ndxview = vws[0] - for v in vws[1:]: - ndxview = ndxview.union(v) - view = view.remapwith(ndxview) - rslt = [] - for row in view: - rslt.append(str(row.id)) - return rslt - - - def list(self): - l = [] - for row in self.getview().select(_isdel=0): - l.append(str(row.id)) - return l - def count(self): - return len(self.getview()) - def getprops(self, protected=1): - # protected is not in ping's spec - allprops = self.ruprops.copy() - if protected and self.privateprops is not None: - allprops.update(self.privateprops) - return allprops - def addprop(self, **properties): - for key in properties.keys(): - if self.ruprops.has_key(key): - raise ValueError, "%s is already a property of %s" % (key, self.classname) - self.ruprops.update(properties) - self.db.fastopen = 0 - view = self.__getview() - self.db.commit() - # ---- end of ping's spec - def filter(self, search_matches, filterspec, sort, group): - # search_matches is None or a set (dict of {nodeid: {propname:[nodeid,...]}}) - # filterspec is a dict {propname:value} - # sort and group are lists of propnames - # sort and group are (dir, prop) where dir is '+', '-' or None - # and prop is a prop name or None - - where = {'_isdel':0} - mlcriteria = {} - regexes = {} - orcriteria = {} - for propname, value in filterspec.items(): - prop = self.ruprops.get(propname, None) - if prop is None: - prop = self.privateprops[propname] - if isinstance(prop, hyperdb.Multilink): - if type(value) is not _LISTTYPE: - value = [value] - # transform keys to ids - u = [] - for item in value: - try: - item = int(item) - except (TypeError, ValueError): - item = int(self.db.getclass(prop.classname).lookup(item)) - if item == -1: - item = 0 - u.append(item) - mlcriteria[propname] = u - elif isinstance(prop, hyperdb.Link): - if type(value) is not _LISTTYPE: - value = [value] - # transform keys to ids - u = [] - for item in value: - try: - item = int(item) - except (TypeError, ValueError): - item = int(self.db.getclass(prop.classname).lookup(item)) - if item == -1: - item = 0 - u.append(item) - if len(u) == 1: - where[propname] = u[0] - else: - orcriteria[propname] = u - elif isinstance(prop, hyperdb.String): - # simple glob searching - v = re.sub(r'([\|\{\}\\\.\+\[\]\(\)])', r'\\\1', value) - v = v.replace('?', '.') - v = v.replace('*', '.*?') - regexes[propname] = re.compile(v, re.I) - elif propname == 'id': - where[propname] = int(value) - elif isinstance(prop, hyperdb.Boolean): - if type(value) is _STRINGTYPE: - bv = value.lower() in ('yes', 'true', 'on', '1') - else: - bv = value - where[propname] = bv - elif isinstance(prop, hyperdb.Number): - where[propname] = int(value) - else: - where[propname] = str(value) - v = self.getview() - #print "filter start at %s" % time.time() - if where: - v = v.select(where) - #print "filter where at %s" % time.time() - - if mlcriteria: - # multilink - if any of the nodeids required by the - # filterspec aren't in this node's property, then skip - # it - def ff(row, ml=mlcriteria): - for propname, values in ml.items(): - sv = getattr(row, propname) - for id in values: - if sv.find(fid=id) == -1: - return 0 - return 1 - iv = v.filter(ff) - v = v.remapwith(iv) - - #print "filter mlcrit at %s" % time.time() - - if orcriteria: - def ff(row, crit=orcriteria): - for propname, allowed in crit.items(): - val = getattr(row, propname) - if val not in allowed: - return 0 - return 1 - - iv = v.filter(ff) - v = v.remapwith(iv) - - #print "filter orcrit at %s" % time.time() - if regexes: - def ff(row, r=regexes): - for propname, regex in r.items(): - val = getattr(row, propname) - if not regex.search(val): - return 0 - return 1 - - iv = v.filter(ff) - v = v.remapwith(iv) - #print "filter regexs at %s" % time.time() - - if sort or group: - sortspec = [] - rev = [] - for dir, propname in group, sort: - if propname is None: continue - isreversed = 0 - if dir == '-': - isreversed = 1 - try: - prop = getattr(v, propname) - except AttributeError: - print "MK has no property %s" % propname - continue - propclass = self.ruprops.get(propname, None) - if propclass is None: - propclass = self.privateprops.get(propname, None) - if propclass is None: - print "Schema has no property %s" % propname - continue - if isinstance(propclass, hyperdb.Link): - linkclass = self.db.getclass(propclass.classname) - lv = linkclass.getview() - lv = lv.rename('id', propname) - v = v.join(lv, prop, 1) - if linkclass.getprops().has_key('order'): - propname = 'order' - else: - propname = linkclass.labelprop() - prop = getattr(v, propname) - if isreversed: - rev.append(prop) - sortspec.append(prop) - v = v.sortrev(sortspec, rev)[:] #XXX Metakit bug - #print "filter sort at %s" % time.time() - - rslt = [] - for row in v: - id = str(row.id) - if search_matches is not None: - if search_matches.has_key(id): - rslt.append(id) - else: - rslt.append(id) - return rslt - - def hasnode(self, nodeid): - return int(nodeid) < self.maxid - - def labelprop(self, default_to_id=0): - ''' Return the property name for a label for the given node. - - This method attempts to generate a consistent label for the node. - It tries the following in order: - 1. key property - 2. "name" property - 3. "title" property - 4. first property from the sorted property name list - ''' - k = self.getkey() - if k: - return k - props = self.getprops() - if props.has_key('name'): - return 'name' - elif props.has_key('title'): - return 'title' - if default_to_id: - return 'id' - props = props.keys() - props.sort() - return props[0] - def stringFind(self, **requirements): - """Locate a particular node by matching a set of its String - properties in a caseless search. - - If the property is not a String property, a TypeError is raised. - - The return is a list of the id of all nodes that match. - """ - for propname in requirements.keys(): - prop = self.properties[propname] - if isinstance(not prop, hyperdb.String): - raise TypeError, "'%s' not a String property"%propname - requirements[propname] = requirements[propname].lower() - requirements['_isdel'] = 0 - - l = [] - for row in self.getview().select(requirements): - l.append(str(row.id)) - return l - - def addjournal(self, nodeid, action, params): - self.db.addjournal(self.classname, nodeid, action, params) - - def index(self, nodeid): - ''' Add (or refresh) the node to search indexes ''' - # find all the String properties that have indexme - for prop, propclass in self.getprops().items(): - if isinstance(propclass, hyperdb.String) and propclass.indexme: - # index them under (classname, nodeid, property) - self.db.indexer.add_text((self.classname, nodeid, prop), - str(self.get(nodeid, prop))) - - def export_list(self, propnames, nodeid): - ''' Export a node - generate a list of CSV-able data in the order - specified by propnames for the given node. - ''' - properties = self.getprops() - l = [] - for prop in propnames: - proptype = properties[prop] - value = self.get(nodeid, prop) - # "marshal" data where needed - if value is None: - pass - elif isinstance(proptype, hyperdb.Date): - value = value.get_tuple() - elif isinstance(proptype, hyperdb.Interval): - value = value.get_tuple() - elif isinstance(proptype, hyperdb.Password): - value = str(value) - l.append(repr(value)) - return l - - def import_list(self, propnames, proplist): - ''' Import a node - all information including "id" is present and - should not be sanity checked. Triggers are not triggered. The - journal should be initialised using the "creator" and "creation" - information. - - Return the nodeid of the node imported. - ''' - if self.db.journaltag is None: - raise hyperdb.DatabaseError, 'Database open read-only' - properties = self.getprops() - - d = {} - view = self.getview(1) - for i in range(len(propnames)): - value = eval(proplist[i]) - propname = propnames[i] - prop = properties[propname] - if propname == 'id': - newid = value - value = int(value) - elif isinstance(prop, hyperdb.Date): - value = int(calendar.timegm(value)) - elif isinstance(prop, hyperdb.Interval): - value = str(date.Interval(value)) - d[propname] = value - view.append(d) - creator = d.get('creator', None) - creation = d.get('creation', None) - self.db.addjournal(self.classname, newid, 'create', {}, creator, creation) - return newid - - # --- used by Database - def _commit(self): - """ called post commit of the DB. - interested subclasses may override """ - self.uncommitted = {} - self.rbactions = [] - self.idcache = {} - def _rollback(self): - """ called pre rollback of the DB. - interested subclasses may override """ - for action in self.rbactions: - action() - self.rbactions = [] - self.uncommitted = {} - self.idcache = {} - def _clear(self): - view = self.getview(1) - if len(view): - view[:] = [] - self.db.dirty = 1 - iv = self.getindexview(1) - if iv: - iv[:] = [] - def rollbackaction(self, action): - """ call this to register a callback called on rollback - callback is removed on end of transaction """ - self.rbactions.append(action) - # --- internal - def __getview(self): - db = self.db._db - view = db.view(self.classname) - mkprops = view.structure() - if mkprops and self.db.fastopen: - return view.ordered(1) - # is the definition the same? - for nm, rutyp in self.ruprops.items(): - for mkprop in mkprops: - if mkprop.name == nm: - break - else: - mkprop = None - if mkprop is None: - break - if _typmap[rutyp.__class__] != mkprop.type: - break - else: - return view.ordered(1) - # need to create or restructure the mk view - # id comes first, so MK will order it for us - self.db.dirty = 1 - s = ["%s[id:I" % self.classname] - for nm, rutyp in self.ruprops.items(): - mktyp = _typmap[rutyp.__class__] - s.append('%s:%s' % (nm, mktyp)) - if mktyp == 'V': - s[-1] += ('[fid:I]') - s.append('_isdel:I,activity:I,creation:I,creator:I]') - v = self.db._db.getas(','.join(s)) - self.db.commit() - return v.ordered(1) - def getview(self, RW=0): - return self.db._db.view(self.classname).ordered(1) - def getindexview(self, RW=0): - return self.db._db.view("_%s" % self.classname).ordered(1) - -def _fetchML(sv): - l = [] - for row in sv: - if row.fid: - l.append(str(row.fid)) - return l - -def _fetchPW(s): - p = password.Password() - p.unpack(s) - return p - -def _fetchLink(n): - return n and str(n) or None - -def _fetchDate(n): - return date.Date(time.gmtime(n)) - -_converters = { - hyperdb.Date : _fetchDate, - hyperdb.Link : _fetchLink, - hyperdb.Multilink : _fetchML, - hyperdb.Interval : date.Interval, - hyperdb.Password : _fetchPW, - hyperdb.Boolean : lambda n: n, - hyperdb.Number : lambda n: n, - hyperdb.String : str, -} - -class FileName(hyperdb.String): - isfilename = 1 - -_typmap = { - FileName : 'S', - hyperdb.String : 'S', - hyperdb.Date : 'I', - hyperdb.Link : 'I', - hyperdb.Multilink : 'V', - hyperdb.Interval : 'S', - hyperdb.Password : 'S', - hyperdb.Boolean : 'I', - hyperdb.Number : 'I', -} -class FileClass(Class): - ' like Class but with a content property ' - default_mime_type = 'text/plain' - def __init__(self, db, classname, **properties): - properties['content'] = FileName() - if not properties.has_key('type'): - properties['type'] = hyperdb.String() - Class.__init__(self, db, classname, **properties) - def get(self, nodeid, propname, default=_marker, cache=1): - x = Class.get(self, nodeid, propname, default, cache) - if propname == 'content': - if x.startswith('file:'): - fnm = x[5:] - try: - x = open(fnm, 'rb').read() - except Exception, e: - x = repr(e) - return x - def create(self, **propvalues): - content = propvalues['content'] - del propvalues['content'] - newid = Class.create(self, **propvalues) - if not content: - return newid - nm = bnm = '%s%s' % (self.classname, newid) - sd = str(int(int(newid) / 1000)) - d = os.path.join(self.db.config.DATABASE, 'files', self.classname, sd) - if not os.path.exists(d): - os.makedirs(d) - nm = os.path.join(d, nm) - open(nm, 'wb').write(content) - self.set(newid, content = 'file:'+nm) - mimetype = propvalues.get('type', self.default_mime_type) - self.db.indexer.add_text((self.classname, newid, 'content'), content, mimetype) - def undo(fnm=nm, action1=os.remove, indexer=self.db.indexer): - action1(fnm) - self.rollbackaction(undo) - return newid - def index(self, nodeid): - Class.index(self, nodeid) - mimetype = self.get(nodeid, 'type') - if not mimetype: - mimetype = self.default_mime_type - self.db.indexer.add_text((self.classname, nodeid, 'content'), - self.get(nodeid, 'content'), mimetype) - -class IssueClass(Class, roundupdb.IssueClass): - # Overridden methods: - def __init__(self, db, classname, **properties): - """The newly-created class automatically includes the "messages", - "files", "nosy", and "superseder" properties. If the 'properties' - dictionary attempts to specify any of these properties or a - "creation" or "activity" property, a ValueError is raised.""" - if not properties.has_key('title'): - properties['title'] = hyperdb.String(indexme='yes') - if not properties.has_key('messages'): - properties['messages'] = hyperdb.Multilink("msg") - if not properties.has_key('files'): - properties['files'] = hyperdb.Multilink("file") - if not properties.has_key('nosy'): - # note: journalling is turned off as it really just wastes - # space. this behaviour may be overridden in an instance - properties['nosy'] = hyperdb.Multilink("user", do_journal="no") - if not properties.has_key('superseder'): - properties['superseder'] = hyperdb.Multilink(classname) - Class.__init__(self, db, classname, **properties) - -CURVERSION = 1 - -class Indexer(indexer.Indexer): - disallows = {'THE':1, 'THIS':1, 'ZZZ':1, 'THAT':1, 'WITH':1} - def __init__(self, path, datadb): - self.db = metakit.storage(os.path.join(path, 'index.mk4'), 1) - self.datadb = datadb - self.reindex = 0 - v = self.db.view('version') - if not v.structure(): - v = self.db.getas('version[vers:I]') - self.db.commit() - v.append(vers=CURVERSION) - self.reindex = 1 - elif v[0].vers != CURVERSION: - v[0].vers = CURVERSION - self.reindex = 1 - if self.reindex: - self.db.getas('ids[tblid:I,nodeid:I,propid:I]') - self.db.getas('index[word:S,hits[pos:I]]') - self.db.commit() - self.reindex = 1 - self.changed = 0 - self.propcache = {} - def force_reindex(self): - v = self.db.view('ids') - v[:] = [] - v = self.db.view('index') - v[:] = [] - self.db.commit() - self.reindex = 1 - def should_reindex(self): - return self.reindex - def _getprops(self, classname): - props = self.propcache.get(classname, None) - if props is None: - props = self.datadb.view(classname).structure() - props = [prop.name for prop in props] - self.propcache[classname] = props - return props - def _getpropid(self, classname, propname): - return self._getprops(classname).index(propname) - def _getpropname(self, classname, propid): - return self._getprops(classname)[propid] - - def add_text(self, identifier, text, mime_type='text/plain'): - if mime_type != 'text/plain': - return - classname, nodeid, property = identifier - tbls = self.datadb.view('tables') - tblid = tbls.find(name=classname) - if tblid < 0: - raise KeyError, "unknown class %r"%classname - nodeid = int(nodeid) - propid = self._getpropid(classname, property) - pos = self.db.view('ids').append(tblid=tblid,nodeid=nodeid,propid=propid) - - wordlist = re.findall(r'\b\w{3,25}\b', text) - words = {} - for word in wordlist: - word = word.upper() - if not self.disallows.has_key(word): - words[word] = 1 - words = words.keys() - - index = self.db.view('index').ordered(1) - for word in words: - ndx = index.find(word=word) - if ndx < 0: - ndx = index.append(word=word) - hits = index[ndx].hits - if len(hits)==0 or hits.find(pos=pos) < 0: - hits.append(pos=pos) - self.changed = 1 - - def find(self, wordlist): - hits = None - index = self.db.view('index').ordered(1) - for word in wordlist: - if not 2 < len(word) < 26: - continue - ndx = index.find(word=word) - if ndx < 0: - return {} - if hits is None: - hits = index[ndx].hits - else: - hits = hits.intersect(index[ndx].hits) - if len(hits) == 0: - return {} - if hits is None: - return {} - rslt = {} - ids = self.db.view('ids').remapwith(hits) - tbls = self.datadb.view('tables') - for i in range(len(ids)): - hit = ids[i] - classname = tbls[hit.tblid].name - nodeid = str(hit.nodeid) - property = self._getpropname(classname, hit.propid) - rslt[i] = (classname, nodeid, property) - return rslt - def save_index(self): - if self.changed: - self.db.commit() - self.changed = 0
--- a/roundup/backends/back_sqlite.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,134 +0,0 @@ -# $Id: back_sqlite.py,v 1.5 2002-09-24 01:59:28 richard Exp $ -__doc__ = ''' -See https://pysqlite.sourceforge.net/ for pysqlite info -''' -import base64, marshal -from roundup.backends.rdbms_common import * -import sqlite - -class Database(Database): - # char to use for positional arguments - arg = '%s' - - def open_connection(self): - # ensure files are group readable and writable - os.umask(0002) - db = os.path.join(self.config.DATABASE, 'db') - self.conn = sqlite.connect(db=db) - self.cursor = self.conn.cursor() - try: - self.database_schema = self.load_dbschema() - except sqlite.DatabaseError, error: - if str(error) != 'no such table: schema': - raise - self.database_schema = {} - self.cursor.execute('create table schema (schema varchar)') - self.cursor.execute('create table ids (name varchar, num integer)') - - def __repr__(self): - return '<roundlite 0x%x>'%id(self) - - def sql_fetchone(self): - ''' Fetch a single row. If there's nothing to fetch, return None. - ''' - return self.cursor.fetchone() - - def sql_fetchall(self): - ''' Fetch a single row. If there's nothing to fetch, return []. - ''' - return self.cursor.fetchall() - - def sql_commit(self): - ''' Actually commit to the database. - - Ignore errors if there's nothing to commit. - ''' - try: - self.conn.commit() - except sqlite.DatabaseError, error: - if str(error) != 'cannot commit - no transaction is active': - raise - - def save_dbschema(self, schema): - ''' Save the schema definition that the database currently implements - ''' - s = repr(self.database_schema) - self.sql('insert into schema values (%s)', (s,)) - - def load_dbschema(self): - ''' Load the schema definition that the database currently implements - ''' - self.cursor.execute('select schema from schema') - return eval(self.cursor.fetchone()[0]) - - def save_journal(self, classname, cols, nodeid, journaldate, - journaltag, action, params): - ''' Save the journal entry to the database - ''' - # make the params db-friendly - params = repr(params) - entry = (nodeid, journaldate, journaltag, action, params) - - # do the insert - a = self.arg - sql = 'insert into %s__journal (%s) values (%s,%s,%s,%s,%s)'%(classname, - cols, a, a, a, a, a) - if __debug__: - print >>hyperdb.DEBUG, 'addjournal', (self, sql, entry) - self.cursor.execute(sql, entry) - - def load_journal(self, classname, cols, nodeid): - ''' Load the journal from the database - ''' - # now get the journal entries - sql = 'select %s from %s__journal where nodeid=%s'%(cols, classname, - self.arg) - if __debug__: - print >>hyperdb.DEBUG, 'getjournal', (self, sql, nodeid) - self.cursor.execute(sql, (nodeid,)) - res = [] - for nodeid, date_stamp, user, action, params in self.cursor.fetchall(): - params = eval(params) - res.append((nodeid, date.Date(date_stamp), user, action, params)) - return res - - def unserialise(self, classname, node): - ''' Decode the marshalled node data - - SQLite stringifies _everything_... so we need to re-numberificate - Booleans and Numbers. - ''' - if __debug__: - print >>hyperdb.DEBUG, 'unserialise', classname, node - properties = self.getclass(classname).getprops() - d = {} - for k, v in node.items(): - # if the property doesn't exist, or is the "retired" flag then - # it won't be in the properties dict - if not properties.has_key(k): - d[k] = v - continue - - # get the property spec - prop = properties[k] - - if isinstance(prop, Date) and v is not None: - d[k] = date.Date(v) - elif isinstance(prop, Interval) and v is not None: - d[k] = date.Interval(v) - elif isinstance(prop, Password): - p = password.Password() - p.unpack(v) - d[k] = p - elif isinstance(prop, Boolean) and v is not None: - d[k] = int(v) - elif isinstance(prop, Number) and v is not None: - # try int first, then assume it's a float - try: - d[k] = int(v) - except ValueError: - d[k] = float(v) - else: - d[k] = v - return d -
--- a/roundup/backends/portalocker.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,93 +0,0 @@ -# portalocker.py - Cross-platform (posix/nt) API for flock-style file locking. -# Requires python 1.5.2 or better. - -# ID line added by richard for Roundup file tracking -# $Id: portalocker.py,v 1.1 2002-04-15 23:25:15 richard Exp $ - -"""Cross-platform (posix/nt) API for flock-style file locking. - -Synopsis: - - import portalocker - file = open("somefile", "r+") - portalocker.lock(file, portalocker.LOCK_EX) - file.seek(12) - file.write("foo") - file.close() - -If you know what you're doing, you may choose to - - portalocker.unlock(file) - -before closing the file, but why? - -Methods: - - lock( file, flags ) - unlock( file ) - -Constants: - - LOCK_EX - LOCK_SH - LOCK_NB - -I learned the win32 technique for locking files from sample code -provided by John Nielsen <nielsenjf@my-deja.com> in the documentation -that accompanies the win32 modules. - -Author: Jonathan Feinberg <jdf@pobox.com> -Version: Id: portalocker.py,v 1.3 2001/05/29 18:47:55 Administrator Exp - **un-cvsified by richard so the version doesn't change** -""" -import os - -if os.name == 'nt': - import win32con - import win32file - import pywintypes - LOCK_EX = win32con.LOCKFILE_EXCLUSIVE_LOCK - LOCK_SH = 0 # the default - LOCK_NB = win32con.LOCKFILE_FAIL_IMMEDIATELY - # is there any reason not to reuse the following structure? - __overlapped = pywintypes.OVERLAPPED() -elif os.name == 'posix': - import fcntl - LOCK_EX = fcntl.LOCK_EX - LOCK_SH = fcntl.LOCK_SH - LOCK_NB = fcntl.LOCK_NB -else: - raise RuntimeError("PortaLocker only defined for nt and posix platforms") - -if os.name == 'nt': - def lock(file, flags): - hfile = win32file._get_osfhandle(file.fileno()) - win32file.LockFileEx(hfile, flags, 0, 0xffff0000, __overlapped) - - def unlock(file): - hfile = win32file._get_osfhandle(file.fileno()) - win32file.UnlockFileEx(hfile, 0, 0xffff0000, __overlapped) - -elif os.name =='posix': - def lock(file, flags): - fcntl.flock(file.fileno(), flags) - - def unlock(file): - fcntl.flock(file.fileno(), fcntl.LOCK_UN) - -if __name__ == '__main__': - from time import time, strftime, localtime - import sys - import portalocker - - log = open('log.txt', "a+") - portalocker.lock(log, portalocker.LOCK_EX) - - timestamp = strftime("%m/%d/%Y %H:%M:%S\n", localtime(time())) - log.write( timestamp ) - - print "Wrote lines. Hit enter to release lock." - dummy = sys.stdin.readline() - - log.close() -
--- a/roundup/backends/rdbms_common.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,2022 +0,0 @@ -# $Id: rdbms_common.py,v 1.19 2002-09-26 03:04:24 richard Exp $ - -# standard python modules -import sys, os, time, re, errno, weakref, copy - -# roundup modules -from roundup import hyperdb, date, password, roundupdb, security -from roundup.hyperdb import String, Password, Date, Interval, Link, \ - Multilink, DatabaseError, Boolean, Number - -# support -from blobfiles import FileStorage -from roundup.indexer import Indexer -from sessions import Sessions - -# number of rows to keep in memory -ROW_CACHE_SIZE = 100 - -class Database(FileStorage, hyperdb.Database, roundupdb.Database): - ''' Wrapper around an SQL database that presents a hyperdb interface. - - - some functionality is specific to the actual SQL database, hence - the sql_* methods that are NotImplemented - - we keep a cache of the latest ROW_CACHE_SIZE row fetches. - ''' - def __init__(self, config, journaltag=None): - ''' Open the database and load the schema from it. - ''' - self.config, self.journaltag = config, journaltag - self.dir = config.DATABASE - self.classes = {} - self.indexer = Indexer(self.dir) - self.sessions = Sessions(self.config) - self.security = security.Security(self) - - # additional transaction support for external files and the like - self.transactions = [] - - # keep a cache of the N most recently retrieved rows of any kind - # (classname, nodeid) = row - self.cache = {} - self.cache_lru = [] - - # open a connection to the database, creating the "conn" attribute - self.open_connection() - - def clearCache(self): - self.cache = {} - self.cache_lru = [] - - def open_connection(self): - ''' Open a connection to the database, creating it if necessary - ''' - raise NotImplemented - - def sql(self, sql, args=None): - ''' Execute the sql with the optional args. - ''' - if __debug__: - print >>hyperdb.DEBUG, (self, sql, args) - if args: - self.cursor.execute(sql, args) - else: - self.cursor.execute(sql) - - def sql_fetchone(self): - ''' Fetch a single row. If there's nothing to fetch, return None. - ''' - raise NotImplemented - - def sql_stringquote(self, value): - ''' Quote the string so it's safe to put in the 'sql quotes' - ''' - return re.sub("'", "''", str(value)) - - def save_dbschema(self, schema): - ''' Save the schema definition that the database currently implements - ''' - raise NotImplemented - - def load_dbschema(self): - ''' Load the schema definition that the database currently implements - ''' - raise NotImplemented - - def post_init(self): - ''' Called once the schema initialisation has finished. - - We should now confirm that the schema defined by our "classes" - attribute actually matches the schema in the database. - ''' - # now detect changes in the schema - save = 0 - for classname, spec in self.classes.items(): - if self.database_schema.has_key(classname): - dbspec = self.database_schema[classname] - if self.update_class(spec, dbspec): - self.database_schema[classname] = spec.schema() - save = 1 - else: - self.create_class(spec) - self.database_schema[classname] = spec.schema() - save = 1 - - for classname in self.database_schema.keys(): - if not self.classes.has_key(classname): - self.drop_class(classname) - - # update the database version of the schema - if save: - self.sql('delete from schema') - self.save_dbschema(self.database_schema) - - # reindex the db if necessary - if self.indexer.should_reindex(): - self.reindex() - - # commit - self.conn.commit() - - # figure the "curuserid" - if self.journaltag is None: - self.curuserid = None - elif self.journaltag == 'admin': - # admin user may not exist, but always has ID 1 - self.curuserid = '1' - else: - self.curuserid = self.user.lookup(self.journaltag) - - def reindex(self): - for klass in self.classes.values(): - for nodeid in klass.list(): - klass.index(nodeid) - self.indexer.save_index() - - def determine_columns(self, properties): - ''' Figure the column names and multilink properties from the spec - - "properties" is a list of (name, prop) where prop may be an - instance of a hyperdb "type" _or_ a string repr of that type. - ''' - cols = ['_activity', '_creator', '_creation'] - mls = [] - # add the multilinks separately - for col, prop in properties: - if isinstance(prop, Multilink): - mls.append(col) - elif isinstance(prop, type('')) and prop.find('Multilink') != -1: - mls.append(col) - else: - cols.append('_'+col) - cols.sort() - return cols, mls - - def update_class(self, spec, dbspec): - ''' Determine the differences between the current spec and the - database version of the spec, and update where necessary - ''' - spec_schema = spec.schema() - if spec_schema == dbspec: - # no save needed for this one - return 0 - if __debug__: - print >>hyperdb.DEBUG, 'update_class FIRING' - - # key property changed? - if dbspec[0] != spec_schema[0]: - if __debug__: - print >>hyperdb.DEBUG, 'update_class setting keyprop', `spec[0]` - # XXX turn on indexing for the key property - - # dict 'em up - spec_propnames,spec_props = [],{} - for propname,prop in spec_schema[1]: - spec_propnames.append(propname) - spec_props[propname] = prop - dbspec_propnames,dbspec_props = [],{} - for propname,prop in dbspec[1]: - dbspec_propnames.append(propname) - dbspec_props[propname] = prop - - # now compare - for propname in spec_propnames: - prop = spec_props[propname] - if dbspec_props.has_key(propname) and prop==dbspec_props[propname]: - continue - if __debug__: - print >>hyperdb.DEBUG, 'update_class ADD', (propname, prop) - - if not dbspec_props.has_key(propname): - # add the property - if isinstance(prop, Multilink): - # all we have to do here is create a new table, easy! - self.create_multilink_table(spec, propname) - continue - - # no ALTER TABLE, so we: - # 1. pull out the data, including an extra None column - oldcols, x = self.determine_columns(dbspec[1]) - oldcols.append('id') - oldcols.append('__retired__') - cn = spec.classname - sql = 'select %s,%s from _%s'%(','.join(oldcols), self.arg, cn) - if __debug__: - print >>hyperdb.DEBUG, 'update_class', (self, sql, None) - self.cursor.execute(sql, (None,)) - olddata = self.cursor.fetchall() - - # 2. drop the old table - self.cursor.execute('drop table _%s'%cn) - - # 3. create the new table - cols, mls = self.create_class_table(spec) - # ensure the new column is last - cols.remove('_'+propname) - assert oldcols == cols, "Column lists don't match!" - cols.append('_'+propname) - - # 4. populate with the data from step one - s = ','.join([self.arg for x in cols]) - scols = ','.join(cols) - sql = 'insert into _%s (%s) values (%s)'%(cn, scols, s) - - # GAH, nothing had better go wrong from here on in... but - # we have to commit the drop... - # XXX this isn't necessary in sqlite :( - self.conn.commit() - - # do the insert - for row in olddata: - self.sql(sql, tuple(row)) - - else: - # modify the property - if __debug__: - print >>hyperdb.DEBUG, 'update_class NOOP' - pass # NOOP in gadfly - - # and the other way - only worry about deletions here - for propname in dbspec_propnames: - prop = dbspec_props[propname] - if spec_props.has_key(propname): - continue - if __debug__: - print >>hyperdb.DEBUG, 'update_class REMOVE', `prop` - - # delete the property - if isinstance(prop, Multilink): - sql = 'drop table %s_%s'%(spec.classname, prop) - if __debug__: - print >>hyperdb.DEBUG, 'update_class', (self, sql) - self.cursor.execute(sql) - else: - # no ALTER TABLE, so we: - # 1. pull out the data, excluding the removed column - oldcols, x = self.determine_columns(spec.properties.items()) - oldcols.append('id') - oldcols.append('__retired__') - # remove the missing column - oldcols.remove('_'+propname) - cn = spec.classname - sql = 'select %s from _%s'%(','.join(oldcols), cn) - self.cursor.execute(sql, (None,)) - olddata = sql.fetchall() - - # 2. drop the old table - self.cursor.execute('drop table _%s'%cn) - - # 3. create the new table - cols, mls = self.create_class_table(self, spec) - assert oldcols != cols, "Column lists don't match!" - - # 4. populate with the data from step one - qs = ','.join([self.arg for x in cols]) - sql = 'insert into _%s values (%s)'%(cn, s) - self.cursor.execute(sql, olddata) - return 1 - - def create_class_table(self, spec): - ''' create the class table for the given spec - ''' - cols, mls = self.determine_columns(spec.properties.items()) - - # add on our special columns - cols.append('id') - cols.append('__retired__') - - # create the base table - scols = ','.join(['%s varchar'%x for x in cols]) - sql = 'create table _%s (%s)'%(spec.classname, scols) - if __debug__: - print >>hyperdb.DEBUG, 'create_class', (self, sql) - self.cursor.execute(sql) - - return cols, mls - - def create_journal_table(self, spec): - ''' create the journal table for a class given the spec and - already-determined cols - ''' - # journal table - cols = ','.join(['%s varchar'%x - for x in 'nodeid date tag action params'.split()]) - sql = 'create table %s__journal (%s)'%(spec.classname, cols) - if __debug__: - print >>hyperdb.DEBUG, 'create_class', (self, sql) - self.cursor.execute(sql) - - def create_multilink_table(self, spec, ml): - ''' Create a multilink table for the "ml" property of the class - given by the spec - ''' - sql = 'create table %s_%s (linkid varchar, nodeid varchar)'%( - spec.classname, ml) - if __debug__: - print >>hyperdb.DEBUG, 'create_class', (self, sql) - self.cursor.execute(sql) - - def create_class(self, spec): - ''' Create a database table according to the given spec. - ''' - cols, mls = self.create_class_table(spec) - self.create_journal_table(spec) - - # now create the multilink tables - for ml in mls: - self.create_multilink_table(spec, ml) - - # ID counter - sql = 'insert into ids (name, num) values (%s,%s)'%(self.arg, self.arg) - vals = (spec.classname, 1) - if __debug__: - print >>hyperdb.DEBUG, 'create_class', (self, sql, vals) - self.cursor.execute(sql, vals) - - def drop_class(self, spec): - ''' Drop the given table from the database. - - Drop the journal and multilink tables too. - ''' - # figure the multilinks - mls = [] - for col, prop in spec.properties.items(): - if isinstance(prop, Multilink): - mls.append(col) - - sql = 'drop table _%s'%spec.classname - if __debug__: - print >>hyperdb.DEBUG, 'drop_class', (self, sql) - self.cursor.execute(sql) - - sql = 'drop table %s__journal'%spec.classname - if __debug__: - print >>hyperdb.DEBUG, 'drop_class', (self, sql) - self.cursor.execute(sql) - - for ml in mls: - sql = 'drop table %s_%s'%(spec.classname, ml) - if __debug__: - print >>hyperdb.DEBUG, 'drop_class', (self, sql) - self.cursor.execute(sql) - - # - # Classes - # - def __getattr__(self, classname): - ''' A convenient way of calling self.getclass(classname). - ''' - if self.classes.has_key(classname): - if __debug__: - print >>hyperdb.DEBUG, '__getattr__', (self, classname) - return self.classes[classname] - raise AttributeError, classname - - def addclass(self, cl): - ''' Add a Class to the hyperdatabase. - ''' - if __debug__: - print >>hyperdb.DEBUG, 'addclass', (self, cl) - cn = cl.classname - if self.classes.has_key(cn): - raise ValueError, cn - self.classes[cn] = cl - - def getclasses(self): - ''' Return a list of the names of all existing classes. - ''' - if __debug__: - print >>hyperdb.DEBUG, 'getclasses', (self,) - l = self.classes.keys() - l.sort() - return l - - def getclass(self, classname): - '''Get the Class object representing a particular class. - - If 'classname' is not a valid class name, a KeyError is raised. - ''' - if __debug__: - print >>hyperdb.DEBUG, 'getclass', (self, classname) - try: - return self.classes[classname] - except KeyError: - raise KeyError, 'There is no class called "%s"'%classname - - def clear(self): - ''' Delete all database contents. - - Note: I don't commit here, which is different behaviour to the - "nuke from orbit" behaviour in the *dbms. - ''' - if __debug__: - print >>hyperdb.DEBUG, 'clear', (self,) - for cn in self.classes.keys(): - sql = 'delete from _%s'%cn - if __debug__: - print >>hyperdb.DEBUG, 'clear', (self, sql) - self.cursor.execute(sql) - - # - # Node IDs - # - def newid(self, classname): - ''' Generate a new id for the given class - ''' - # get the next ID - sql = 'select num from ids where name=%s'%self.arg - if __debug__: - print >>hyperdb.DEBUG, 'newid', (self, sql, classname) - self.cursor.execute(sql, (classname, )) - newid = self.cursor.fetchone()[0] - - # update the counter - sql = 'update ids set num=%s where name=%s'%(self.arg, self.arg) - vals = (int(newid)+1, classname) - if __debug__: - print >>hyperdb.DEBUG, 'newid', (self, sql, vals) - self.cursor.execute(sql, vals) - - # return as string - return str(newid) - - def setid(self, classname, setid): - ''' Set the id counter: used during import of database - ''' - sql = 'update ids set num=%s where name=%s'%(self.arg, self.arg) - vals = (setid, classname) - if __debug__: - print >>hyperdb.DEBUG, 'setid', (self, sql, vals) - self.cursor.execute(sql, vals) - - # - # Nodes - # - - def addnode(self, classname, nodeid, node): - ''' Add the specified node to its class's db. - ''' - if __debug__: - print >>hyperdb.DEBUG, 'addnode', (self, classname, nodeid, node) - # gadfly requires values for all non-multilink columns - cl = self.classes[classname] - cols, mls = self.determine_columns(cl.properties.items()) - - # we'll be supplied these props if we're doing an import - if not node.has_key('creator'): - # add in the "calculated" properties (dupe so we don't affect - # calling code's node assumptions) - node = node.copy() - node['creation'] = node['activity'] = date.Date() - node['creator'] = self.curuserid - - # default the non-multilink columns - for col, prop in cl.properties.items(): - if not isinstance(col, Multilink): - if not node.has_key(col): - node[col] = None - - # clear this node out of the cache if it's in there - key = (classname, nodeid) - if self.cache.has_key(key): - del self.cache[key] - self.cache_lru.remove(key) - - # make the node data safe for the DB - node = self.serialise(classname, node) - - # make sure the ordering is correct for column name -> column value - vals = tuple([node[col[1:]] for col in cols]) + (nodeid, 0) - s = ','.join([self.arg for x in cols]) + ',%s,%s'%(self.arg, self.arg) - cols = ','.join(cols) + ',id,__retired__' - - # perform the inserts - sql = 'insert into _%s (%s) values (%s)'%(classname, cols, s) - if __debug__: - print >>hyperdb.DEBUG, 'addnode', (self, sql, vals) - self.cursor.execute(sql, vals) - - # insert the multilink rows - for col in mls: - t = '%s_%s'%(classname, col) - for entry in node[col]: - sql = 'insert into %s (linkid, nodeid) values (%s,%s)'%(t, - self.arg, self.arg) - self.sql(sql, (entry, nodeid)) - - # make sure we do the commit-time extra stuff for this node - self.transactions.append((self.doSaveNode, (classname, nodeid, node))) - - def setnode(self, classname, nodeid, values, multilink_changes): - ''' Change the specified node. - ''' - if __debug__: - print >>hyperdb.DEBUG, 'setnode', (self, classname, nodeid, values) - - # clear this node out of the cache if it's in there - key = (classname, nodeid) - if self.cache.has_key(key): - del self.cache[key] - self.cache_lru.remove(key) - - # add the special props - values = values.copy() - values['activity'] = date.Date() - - # make db-friendly - values = self.serialise(classname, values) - - cl = self.classes[classname] - cols = [] - mls = [] - # add the multilinks separately - props = cl.getprops() - for col in values.keys(): - prop = props[col] - if isinstance(prop, Multilink): - mls.append(col) - else: - cols.append('_'+col) - cols.sort() - - # if there's any updates to regular columns, do them - if cols: - # make sure the ordering is correct for column name -> column value - sqlvals = tuple([values[col[1:]] for col in cols]) + (nodeid,) - s = ','.join(['%s=%s'%(x, self.arg) for x in cols]) - cols = ','.join(cols) - - # perform the update - sql = 'update _%s set %s where id=%s'%(classname, s, self.arg) - if __debug__: - print >>hyperdb.DEBUG, 'setnode', (self, sql, sqlvals) - self.cursor.execute(sql, sqlvals) - - # now the fun bit, updating the multilinks ;) - for col, (add, remove) in multilink_changes.items(): - tn = '%s_%s'%(classname, col) - if add: - sql = 'insert into %s (nodeid, linkid) values (%s,%s)'%(tn, - self.arg, self.arg) - for addid in add: - self.sql(sql, (nodeid, addid)) - if remove: - sql = 'delete from %s where nodeid=%s and linkid=%s'%(tn, - self.arg, self.arg) - for removeid in remove: - self.sql(sql, (nodeid, removeid)) - - # make sure we do the commit-time extra stuff for this node - self.transactions.append((self.doSaveNode, (classname, nodeid, values))) - - def getnode(self, classname, nodeid): - ''' Get a node from the database. - ''' - if __debug__: - print >>hyperdb.DEBUG, 'getnode', (self, classname, nodeid) - - # see if we have this node cached - key = (classname, nodeid) - if self.cache.has_key(key): - # push us back to the top of the LRU - self.cache_lru.remove(key) - self.cache_lru.insert(0, key) - # return the cached information - return self.cache[key] - - # figure the columns we're fetching - cl = self.classes[classname] - cols, mls = self.determine_columns(cl.properties.items()) - scols = ','.join(cols) - - # perform the basic property fetch - sql = 'select %s from _%s where id=%s'%(scols, classname, self.arg) - self.sql(sql, (nodeid,)) - - values = self.sql_fetchone() - if values is None: - raise IndexError, 'no such %s node %s'%(classname, nodeid) - - # make up the node - node = {} - for col in range(len(cols)): - node[cols[col][1:]] = values[col] - - # now the multilinks - for col in mls: - # get the link ids - sql = 'select linkid from %s_%s where nodeid=%s'%(classname, col, - self.arg) - self.cursor.execute(sql, (nodeid,)) - # extract the first column from the result - node[col] = [x[0] for x in self.cursor.fetchall()] - - # un-dbificate the node data - node = self.unserialise(classname, node) - - # save off in the cache - key = (classname, nodeid) - self.cache[key] = node - # update the LRU - self.cache_lru.insert(0, key) - if len(self.cache_lru) > ROW_CACHE_SIZE: - del self.cache[self.cache_lru.pop()] - - return node - - def destroynode(self, classname, nodeid): - '''Remove a node from the database. Called exclusively by the - destroy() method on Class. - ''' - if __debug__: - print >>hyperdb.DEBUG, 'destroynode', (self, classname, nodeid) - - # make sure the node exists - if not self.hasnode(classname, nodeid): - raise IndexError, '%s has no node %s'%(classname, nodeid) - - # see if we have this node cached - if self.cache.has_key((classname, nodeid)): - del self.cache[(classname, nodeid)] - - # see if there's any obvious commit actions that we should get rid of - for entry in self.transactions[:]: - if entry[1][:2] == (classname, nodeid): - self.transactions.remove(entry) - - # now do the SQL - sql = 'delete from _%s where id=%s'%(classname, self.arg) - self.sql(sql, (nodeid,)) - - # remove from multilnks - cl = self.getclass(classname) - x, mls = self.determine_columns(cl.properties.items()) - for col in mls: - # get the link ids - sql = 'delete from %s_%s where nodeid=%s'%(classname, col, self.arg) - self.cursor.execute(sql, (nodeid,)) - - # remove journal entries - sql = 'delete from %s__journal where nodeid=%s'%(classname, self.arg) - self.sql(sql, (nodeid,)) - - def serialise(self, classname, node): - '''Copy the node contents, converting non-marshallable data into - marshallable data. - ''' - if __debug__: - print >>hyperdb.DEBUG, 'serialise', classname, node - properties = self.getclass(classname).getprops() - d = {} - for k, v in node.items(): - # if the property doesn't exist, or is the "retired" flag then - # it won't be in the properties dict - if not properties.has_key(k): - d[k] = v - continue - - # get the property spec - prop = properties[k] - - if isinstance(prop, Password): - d[k] = str(v) - elif isinstance(prop, Date) and v is not None: - d[k] = v.serialise() - elif isinstance(prop, Interval) and v is not None: - d[k] = v.serialise() - else: - d[k] = v - return d - - def unserialise(self, classname, node): - '''Decode the marshalled node data - ''' - if __debug__: - print >>hyperdb.DEBUG, 'unserialise', classname, node - properties = self.getclass(classname).getprops() - d = {} - for k, v in node.items(): - # if the property doesn't exist, or is the "retired" flag then - # it won't be in the properties dict - if not properties.has_key(k): - d[k] = v - continue - - # get the property spec - prop = properties[k] - - if isinstance(prop, Date) and v is not None: - d[k] = date.Date(v) - elif isinstance(prop, Interval) and v is not None: - d[k] = date.Interval(v) - elif isinstance(prop, Password): - p = password.Password() - p.unpack(v) - d[k] = p - else: - d[k] = v - return d - - def hasnode(self, classname, nodeid): - ''' Determine if the database has a given node. - ''' - sql = 'select count(*) from _%s where id=%s'%(classname, self.arg) - if __debug__: - print >>hyperdb.DEBUG, 'hasnode', (self, sql, nodeid) - self.cursor.execute(sql, (nodeid,)) - return int(self.cursor.fetchone()[0]) - - def countnodes(self, classname): - ''' Count the number of nodes that exist for a particular Class. - ''' - sql = 'select count(*) from _%s'%classname - if __debug__: - print >>hyperdb.DEBUG, 'countnodes', (self, sql) - self.cursor.execute(sql) - return self.cursor.fetchone()[0] - - def getnodeids(self, classname, retired=0): - ''' Retrieve all the ids of the nodes for a particular Class. - - Set retired=None to get all nodes. Otherwise it'll get all the - retired or non-retired nodes, depending on the flag. - ''' - # flip the sense of the flag if we don't want all of them - if retired is not None: - retired = not retired - sql = 'select id from _%s where __retired__ <> %s'%(classname, self.arg) - if __debug__: - print >>hyperdb.DEBUG, 'getnodeids', (self, sql, retired) - self.cursor.execute(sql, (retired,)) - return [x[0] for x in self.cursor.fetchall()] - - def addjournal(self, classname, nodeid, action, params, creator=None, - creation=None): - ''' Journal the Action - 'action' may be: - - 'create' or 'set' -- 'params' is a dictionary of property values - 'link' or 'unlink' -- 'params' is (classname, nodeid, propname) - 'retire' -- 'params' is None - ''' - # serialise the parameters now if necessary - if isinstance(params, type({})): - if action in ('set', 'create'): - params = self.serialise(classname, params) - - # handle supply of the special journalling parameters (usually - # supplied on importing an existing database) - if creator: - journaltag = creator - else: - journaltag = self.curuserid - if creation: - journaldate = creation.serialise() - else: - journaldate = date.Date().serialise() - - # create the journal entry - cols = ','.join('nodeid date tag action params'.split()) - - if __debug__: - print >>hyperdb.DEBUG, 'addjournal', (nodeid, journaldate, - journaltag, action, params) - - self.save_journal(classname, cols, nodeid, journaldate, - journaltag, action, params) - - def save_journal(self, classname, cols, nodeid, journaldate, - journaltag, action, params): - ''' Save the journal entry to the database - ''' - raise NotImplemented - - def getjournal(self, classname, nodeid): - ''' get the journal for id - ''' - # make sure the node exists - if not self.hasnode(classname, nodeid): - raise IndexError, '%s has no node %s'%(classname, nodeid) - - cols = ','.join('nodeid date tag action params'.split()) - return self.load_journal(classname, cols, nodeid) - - def load_journal(self, classname, cols, nodeid): - ''' Load the journal from the database - ''' - raise NotImplemented - - def pack(self, pack_before): - ''' Delete all journal entries except "create" before 'pack_before'. - ''' - # get a 'yyyymmddhhmmss' version of the date - date_stamp = pack_before.serialise() - - # do the delete - for classname in self.classes.keys(): - sql = "delete from %s__journal where date<%s and "\ - "action<>'create'"%(classname, self.arg) - if __debug__: - print >>hyperdb.DEBUG, 'pack', (self, sql, date_stamp) - self.cursor.execute(sql, (date_stamp,)) - - def sql_commit(self): - ''' Actually commit to the database. - ''' - self.conn.commit() - - def commit(self): - ''' Commit the current transactions. - - Save all data changed since the database was opened or since the - last commit() or rollback(). - ''' - if __debug__: - print >>hyperdb.DEBUG, 'commit', (self,) - - # commit the database - self.sql_commit() - - # now, do all the other transaction stuff - reindex = {} - for method, args in self.transactions: - reindex[method(*args)] = 1 - - # reindex the nodes that request it - for classname, nodeid in filter(None, reindex.keys()): - print >>hyperdb.DEBUG, 'commit.reindex', (classname, nodeid) - self.getclass(classname).index(nodeid) - - # save the indexer state - self.indexer.save_index() - - # clear out the transactions - self.transactions = [] - - def rollback(self): - ''' Reverse all actions from the current transaction. - - Undo all the changes made since the database was opened or the last - commit() or rollback() was performed. - ''' - if __debug__: - print >>hyperdb.DEBUG, 'rollback', (self,) - - # roll back - self.conn.rollback() - - # roll back "other" transaction stuff - for method, args in self.transactions: - # delete temporary files - if method == self.doStoreFile: - self.rollbackStoreFile(*args) - self.transactions = [] - - def doSaveNode(self, classname, nodeid, node): - ''' dummy that just generates a reindex event - ''' - # return the classname, nodeid so we reindex this content - return (classname, nodeid) - - def close(self): - ''' Close off the connection. - ''' - self.conn.close() - -# -# The base Class class -# -class Class(hyperdb.Class): - ''' The handle to a particular class of nodes in a hyperdatabase. - - All methods except __repr__ and getnode must be implemented by a - concrete backend Class. - ''' - - def __init__(self, db, classname, **properties): - '''Create a new class with a given name and property specification. - - 'classname' must not collide with the name of an existing class, - or a ValueError is raised. The keyword arguments in 'properties' - must map names to property objects, or a TypeError is raised. - ''' - if (properties.has_key('creation') or properties.has_key('activity') - or properties.has_key('creator')): - raise ValueError, '"creation", "activity" and "creator" are '\ - 'reserved' - - self.classname = classname - self.properties = properties - self.db = weakref.proxy(db) # use a weak ref to avoid circularity - self.key = '' - - # should we journal changes (default yes) - self.do_journal = 1 - - # do the db-related init stuff - db.addclass(self) - - self.auditors = {'create': [], 'set': [], 'retire': []} - self.reactors = {'create': [], 'set': [], 'retire': []} - - def schema(self): - ''' A dumpable version of the schema that we can store in the - database - ''' - return (self.key, [(x, repr(y)) for x,y in self.properties.items()]) - - def enableJournalling(self): - '''Turn journalling on for this class - ''' - self.do_journal = 1 - - def disableJournalling(self): - '''Turn journalling off for this class - ''' - self.do_journal = 0 - - # Editing nodes: - def create(self, **propvalues): - ''' Create a new node of this class and return its id. - - The keyword arguments in 'propvalues' map property names to values. - - The values of arguments must be acceptable for the types of their - corresponding properties or a TypeError is raised. - - If this class has a key property, it must be present and its value - must not collide with other key strings or a ValueError is raised. - - Any other properties on this class that are missing from the - 'propvalues' dictionary are set to None. - - If an id in a link or multilink property does not refer to a valid - node, an IndexError is raised. - ''' - if propvalues.has_key('id'): - raise KeyError, '"id" is reserved' - - if self.db.journaltag is None: - raise DatabaseError, 'Database open read-only' - - if propvalues.has_key('creation') or propvalues.has_key('activity'): - raise KeyError, '"creation" and "activity" are reserved' - - self.fireAuditors('create', None, propvalues) - - # new node's id - newid = self.db.newid(self.classname) - - # validate propvalues - num_re = re.compile('^\d+$') - for key, value in propvalues.items(): - if key == self.key: - try: - self.lookup(value) - except KeyError: - pass - else: - raise ValueError, 'node with key "%s" exists'%value - - # try to handle this property - try: - prop = self.properties[key] - except KeyError: - raise KeyError, '"%s" has no property "%s"'%(self.classname, - key) - - if value is not None and isinstance(prop, Link): - if type(value) != type(''): - raise ValueError, 'link value must be String' - link_class = self.properties[key].classname - # if it isn't a number, it's a key - if not num_re.match(value): - try: - value = self.db.classes[link_class].lookup(value) - except (TypeError, KeyError): - raise IndexError, 'new property "%s": %s not a %s'%( - key, value, link_class) - elif not self.db.getclass(link_class).hasnode(value): - raise IndexError, '%s has no node %s'%(link_class, value) - - # save off the value - propvalues[key] = value - - # register the link with the newly linked node - if self.do_journal and self.properties[key].do_journal: - self.db.addjournal(link_class, value, 'link', - (self.classname, newid, key)) - - elif isinstance(prop, Multilink): - if type(value) != type([]): - raise TypeError, 'new property "%s" not a list of ids'%key - - # clean up and validate the list of links - link_class = self.properties[key].classname - l = [] - for entry in value: - if type(entry) != type(''): - raise ValueError, '"%s" multilink value (%r) '\ - 'must contain Strings'%(key, value) - # if it isn't a number, it's a key - if not num_re.match(entry): - try: - entry = self.db.classes[link_class].lookup(entry) - except (TypeError, KeyError): - raise IndexError, 'new property "%s": %s not a %s'%( - key, entry, self.properties[key].classname) - l.append(entry) - value = l - propvalues[key] = value - - # handle additions - for nodeid in value: - if not self.db.getclass(link_class).hasnode(nodeid): - raise IndexError, '%s has no node %s'%(link_class, - nodeid) - # register the link with the newly linked node - if self.do_journal and self.properties[key].do_journal: - self.db.addjournal(link_class, nodeid, 'link', - (self.classname, newid, key)) - - elif isinstance(prop, String): - if type(value) != type(''): - raise TypeError, 'new property "%s" not a string'%key - - elif isinstance(prop, Password): - if not isinstance(value, password.Password): - raise TypeError, 'new property "%s" not a Password'%key - - elif isinstance(prop, Date): - if value is not None and not isinstance(value, date.Date): - raise TypeError, 'new property "%s" not a Date'%key - - elif isinstance(prop, Interval): - if value is not None and not isinstance(value, date.Interval): - raise TypeError, 'new property "%s" not an Interval'%key - - elif value is not None and isinstance(prop, Number): - try: - float(value) - except ValueError: - raise TypeError, 'new property "%s" not numeric'%key - - elif value is not None and isinstance(prop, Boolean): - try: - int(value) - except ValueError: - raise TypeError, 'new property "%s" not boolean'%key - - # make sure there's data where there needs to be - for key, prop in self.properties.items(): - if propvalues.has_key(key): - continue - if key == self.key: - raise ValueError, 'key property "%s" is required'%key - if isinstance(prop, Multilink): - propvalues[key] = [] - else: - propvalues[key] = None - - # done - self.db.addnode(self.classname, newid, propvalues) - if self.do_journal: - self.db.addjournal(self.classname, newid, 'create', propvalues) - - self.fireReactors('create', newid, None) - - return newid - - def export_list(self, propnames, nodeid): - ''' Export a node - generate a list of CSV-able data in the order - specified by propnames for the given node. - ''' - properties = self.getprops() - l = [] - for prop in propnames: - proptype = properties[prop] - value = self.get(nodeid, prop) - # "marshal" data where needed - if value is None: - pass - elif isinstance(proptype, hyperdb.Date): - value = value.get_tuple() - elif isinstance(proptype, hyperdb.Interval): - value = value.get_tuple() - elif isinstance(proptype, hyperdb.Password): - value = str(value) - l.append(repr(value)) - return l - - def import_list(self, propnames, proplist): - ''' Import a node - all information including "id" is present and - should not be sanity checked. Triggers are not triggered. The - journal should be initialised using the "creator" and "created" - information. - - Return the nodeid of the node imported. - ''' - if self.db.journaltag is None: - raise DatabaseError, 'Database open read-only' - properties = self.getprops() - - # make the new node's property map - d = {} - for i in range(len(propnames)): - # Use eval to reverse the repr() used to output the CSV - value = eval(proplist[i]) - - # Figure the property for this column - propname = propnames[i] - prop = properties[propname] - - # "unmarshal" where necessary - if propname == 'id': - newid = value - continue - elif value is None: - # don't set Nones - continue - elif isinstance(prop, hyperdb.Date): - value = date.Date(value) - elif isinstance(prop, hyperdb.Interval): - value = date.Interval(value) - elif isinstance(prop, hyperdb.Password): - pwd = password.Password() - pwd.unpack(value) - value = pwd - d[propname] = value - - # add the node and journal - self.db.addnode(self.classname, newid, d) - - # extract the extraneous journalling gumpf and nuke it - if d.has_key('creator'): - creator = d['creator'] - del d['creator'] - else: - creator = None - if d.has_key('creation'): - creation = d['creation'] - del d['creation'] - else: - creation = None - if d.has_key('activity'): - del d['activity'] - self.db.addjournal(self.classname, newid, 'create', d, creator, - creation) - return newid - - _marker = [] - def get(self, nodeid, propname, default=_marker, cache=1): - '''Get the value of a property on an existing node of this class. - - 'nodeid' must be the id of an existing node of this class or an - IndexError is raised. 'propname' must be the name of a property - of this class or a KeyError is raised. - - 'cache' indicates whether the transaction cache should be queried - for the node. If the node has been modified and you need to - determine what its values prior to modification are, you need to - set cache=0. - ''' - if propname == 'id': - return nodeid - - # get the node's dict - d = self.db.getnode(self.classname, nodeid) - - if propname == 'creation': - if d.has_key('creation'): - return d['creation'] - else: - return date.Date() - if propname == 'activity': - if d.has_key('activity'): - return d['activity'] - else: - return date.Date() - if propname == 'creator': - if d.has_key('creator'): - return d['creator'] - else: - return self.db.curuserid - - # get the property (raises KeyErorr if invalid) - prop = self.properties[propname] - - if not d.has_key(propname): - if default is self._marker: - if isinstance(prop, Multilink): - return [] - else: - return None - else: - return default - - # don't pass our list to other code - if isinstance(prop, Multilink): - return d[propname][:] - - return d[propname] - - def getnode(self, nodeid, cache=1): - ''' Return a convenience wrapper for the node. - - 'nodeid' must be the id of an existing node of this class or an - IndexError is raised. - - 'cache' indicates whether the transaction cache should be queried - for the node. If the node has been modified and you need to - determine what its values prior to modification are, you need to - set cache=0. - ''' - return Node(self, nodeid, cache=cache) - - def set(self, nodeid, **propvalues): - '''Modify a property on an existing node of this class. - - 'nodeid' must be the id of an existing node of this class or an - IndexError is raised. - - Each key in 'propvalues' must be the name of a property of this - class or a KeyError is raised. - - All values in 'propvalues' must be acceptable types for their - corresponding properties or a TypeError is raised. - - If the value of the key property is set, it must not collide with - other key strings or a ValueError is raised. - - If the value of a Link or Multilink property contains an invalid - node id, a ValueError is raised. - ''' - if not propvalues: - return propvalues - - if propvalues.has_key('creation') or propvalues.has_key('activity'): - raise KeyError, '"creation" and "activity" are reserved' - - if propvalues.has_key('id'): - raise KeyError, '"id" is reserved' - - if self.db.journaltag is None: - raise DatabaseError, 'Database open read-only' - - self.fireAuditors('set', nodeid, propvalues) - # Take a copy of the node dict so that the subsequent set - # operation doesn't modify the oldvalues structure. - # XXX used to try the cache here first - oldvalues = copy.deepcopy(self.db.getnode(self.classname, nodeid)) - - node = self.db.getnode(self.classname, nodeid) - if self.is_retired(nodeid): - raise IndexError, 'Requested item is retired' - num_re = re.compile('^\d+$') - - # if the journal value is to be different, store it in here - journalvalues = {} - - # remember the add/remove stuff for multilinks, making it easier - # for the Database layer to do its stuff - multilink_changes = {} - - for propname, value in propvalues.items(): - # check to make sure we're not duplicating an existing key - if propname == self.key and node[propname] != value: - try: - self.lookup(value) - except KeyError: - pass - else: - raise ValueError, 'node with key "%s" exists'%value - - # this will raise the KeyError if the property isn't valid - # ... we don't use getprops() here because we only care about - # the writeable properties. - try: - prop = self.properties[propname] - except KeyError: - raise KeyError, '"%s" has no property named "%s"'%( - self.classname, propname) - - # if the value's the same as the existing value, no sense in - # doing anything - if node.has_key(propname) and value == node[propname]: - del propvalues[propname] - continue - - # do stuff based on the prop type - if isinstance(prop, Link): - link_class = prop.classname - # if it isn't a number, it's a key - if value is not None and not isinstance(value, type('')): - raise ValueError, 'property "%s" link value be a string'%( - propname) - if isinstance(value, type('')) and not num_re.match(value): - try: - value = self.db.classes[link_class].lookup(value) - except (TypeError, KeyError): - raise IndexError, 'new property "%s": %s not a %s'%( - propname, value, prop.classname) - - if (value is not None and - not self.db.getclass(link_class).hasnode(value)): - raise IndexError, '%s has no node %s'%(link_class, value) - - if self.do_journal and prop.do_journal: - # register the unlink with the old linked node - if node[propname] is not None: - self.db.addjournal(link_class, node[propname], 'unlink', - (self.classname, nodeid, propname)) - - # register the link with the newly linked node - if value is not None: - self.db.addjournal(link_class, value, 'link', - (self.classname, nodeid, propname)) - - elif isinstance(prop, Multilink): - if type(value) != type([]): - raise TypeError, 'new property "%s" not a list of'\ - ' ids'%propname - link_class = self.properties[propname].classname - l = [] - for entry in value: - # if it isn't a number, it's a key - if type(entry) != type(''): - raise ValueError, 'new property "%s" link value ' \ - 'must be a string'%propname - if not num_re.match(entry): - try: - entry = self.db.classes[link_class].lookup(entry) - except (TypeError, KeyError): - raise IndexError, 'new property "%s": %s not a %s'%( - propname, entry, - self.properties[propname].classname) - l.append(entry) - value = l - propvalues[propname] = value - - # figure the journal entry for this property - add = [] - remove = [] - - # handle removals - if node.has_key(propname): - l = node[propname] - else: - l = [] - for id in l[:]: - if id in value: - continue - # register the unlink with the old linked node - if self.do_journal and self.properties[propname].do_journal: - self.db.addjournal(link_class, id, 'unlink', - (self.classname, nodeid, propname)) - l.remove(id) - remove.append(id) - - # handle additions - for id in value: - if not self.db.getclass(link_class).hasnode(id): - raise IndexError, '%s has no node %s'%(link_class, id) - if id in l: - continue - # register the link with the newly linked node - if self.do_journal and self.properties[propname].do_journal: - self.db.addjournal(link_class, id, 'link', - (self.classname, nodeid, propname)) - l.append(id) - add.append(id) - - # figure the journal entry - l = [] - if add: - l.append(('+', add)) - if remove: - l.append(('-', remove)) - multilink_changes[propname] = (add, remove) - if l: - journalvalues[propname] = tuple(l) - - elif isinstance(prop, String): - if value is not None and type(value) != type(''): - raise TypeError, 'new property "%s" not a string'%propname - - elif isinstance(prop, Password): - if not isinstance(value, password.Password): - raise TypeError, 'new property "%s" not a Password'%propname - propvalues[propname] = value - - elif value is not None and isinstance(prop, Date): - if not isinstance(value, date.Date): - raise TypeError, 'new property "%s" not a Date'% propname - propvalues[propname] = value - - elif value is not None and isinstance(prop, Interval): - if not isinstance(value, date.Interval): - raise TypeError, 'new property "%s" not an '\ - 'Interval'%propname - propvalues[propname] = value - - elif value is not None and isinstance(prop, Number): - try: - float(value) - except ValueError: - raise TypeError, 'new property "%s" not numeric'%propname - - elif value is not None and isinstance(prop, Boolean): - try: - int(value) - except ValueError: - raise TypeError, 'new property "%s" not boolean'%propname - - # nothing to do? - if not propvalues: - return propvalues - - # do the set, and journal it - self.db.setnode(self.classname, nodeid, propvalues, multilink_changes) - - if self.do_journal: - propvalues.update(journalvalues) - self.db.addjournal(self.classname, nodeid, 'set', propvalues) - - self.fireReactors('set', nodeid, oldvalues) - - return propvalues - - def retire(self, nodeid): - '''Retire a node. - - The properties on the node remain available from the get() method, - and the node's id is never reused. - - Retired nodes are not returned by the find(), list(), or lookup() - methods, and other nodes may reuse the values of their key properties. - ''' - if self.db.journaltag is None: - raise DatabaseError, 'Database open read-only' - - # use the arg for __retired__ to cope with any odd database type - # conversion (hello, sqlite) - sql = 'update _%s set __retired__=%s where id=%s'%(self.classname, - self.db.arg, self.db.arg) - if __debug__: - print >>hyperdb.DEBUG, 'retire', (self, sql, nodeid) - self.db.cursor.execute(sql, (1, nodeid)) - - def is_retired(self, nodeid): - '''Return true if the node is rerired - ''' - sql = 'select __retired__ from _%s where id=%s'%(self.classname, - self.db.arg) - if __debug__: - print >>hyperdb.DEBUG, 'is_retired', (self, sql, nodeid) - self.db.cursor.execute(sql, (nodeid,)) - return int(self.db.sql_fetchone()[0]) - - def destroy(self, nodeid): - '''Destroy a node. - - WARNING: this method should never be used except in extremely rare - situations where there could never be links to the node being - deleted - WARNING: use retire() instead - WARNING: the properties of this node will not be available ever again - WARNING: really, use retire() instead - - Well, I think that's enough warnings. This method exists mostly to - support the session storage of the cgi interface. - - The node is completely removed from the hyperdb, including all journal - entries. It will no longer be available, and will generally break code - if there are any references to the node. - ''' - if self.db.journaltag is None: - raise DatabaseError, 'Database open read-only' - self.db.destroynode(self.classname, nodeid) - - def history(self, nodeid): - '''Retrieve the journal of edits on a particular node. - - 'nodeid' must be the id of an existing node of this class or an - IndexError is raised. - - The returned list contains tuples of the form - - (date, tag, action, params) - - 'date' is a Timestamp object specifying the time of the change and - 'tag' is the journaltag specified when the database was opened. - ''' - if not self.do_journal: - raise ValueError, 'Journalling is disabled for this class' - return self.db.getjournal(self.classname, nodeid) - - # Locating nodes: - def hasnode(self, nodeid): - '''Determine if the given nodeid actually exists - ''' - return self.db.hasnode(self.classname, nodeid) - - def setkey(self, propname): - '''Select a String property of this class to be the key property. - - 'propname' must be the name of a String property of this class or - None, or a TypeError is raised. The values of the key property on - all existing nodes must be unique or a ValueError is raised. - ''' - # XXX create an index on the key prop column - prop = self.getprops()[propname] - if not isinstance(prop, String): - raise TypeError, 'key properties must be String' - self.key = propname - - def getkey(self): - '''Return the name of the key property for this class or None.''' - return self.key - - def labelprop(self, default_to_id=0): - ''' Return the property name for a label for the given node. - - This method attempts to generate a consistent label for the node. - It tries the following in order: - 1. key property - 2. "name" property - 3. "title" property - 4. first property from the sorted property name list - ''' - k = self.getkey() - if k: - return k - props = self.getprops() - if props.has_key('name'): - return 'name' - elif props.has_key('title'): - return 'title' - if default_to_id: - return 'id' - props = props.keys() - props.sort() - return props[0] - - def lookup(self, keyvalue): - '''Locate a particular node by its key property and return its id. - - If this class has no key property, a TypeError is raised. If the - 'keyvalue' matches one of the values for the key property among - the nodes in this class, the matching node's id is returned; - otherwise a KeyError is raised. - ''' - if not self.key: - raise TypeError, 'No key property set for class %s'%self.classname - - # use the arg to handle any odd database type conversion (hello, - # sqlite) - sql = "select id from _%s where _%s=%s and __retired__ <> %s"%( - self.classname, self.key, self.db.arg, self.db.arg) - self.db.sql(sql, (keyvalue, 1)) - - # see if there was a result that's not retired - row = self.db.sql_fetchone() - if not row: - raise KeyError, 'No key (%s) value "%s" for "%s"'%(self.key, - keyvalue, self.classname) - - # return the id - return row[0] - - def find(self, **propspec): - '''Get the ids of nodes in this class which link to the given nodes. - - 'propspec' consists of keyword args propname=nodeid or - propname={nodeid:1, } - 'propname' must be the name of a property in this class, or a - KeyError is raised. That property must be a Link or Multilink - property, or a TypeError is raised. - - Any node in this class whose 'propname' property links to any of the - nodeids will be returned. Used by the full text indexing, which knows - that "foo" occurs in msg1, msg3 and file7, so we have hits on these - issues: - - db.issue.find(messages={'1':1,'3':1}, files={'7':1}) - ''' - if __debug__: - print >>hyperdb.DEBUG, 'find', (self, propspec) - - # shortcut - if not propspec: - return [] - - # validate the args - props = self.getprops() - propspec = propspec.items() - for propname, nodeids in propspec: - # check the prop is OK - prop = props[propname] - if not isinstance(prop, Link) and not isinstance(prop, Multilink): - raise TypeError, "'%s' not a Link/Multilink property"%propname - - # first, links - where = [] - allvalues = () - a = self.db.arg - for prop, values in propspec: - if not isinstance(props[prop], hyperdb.Link): - continue - if type(values) is type(''): - allvalues += (values,) - where.append('_%s = %s'%(prop, a)) - else: - allvalues += tuple(values.keys()) - where.append('_%s in (%s)'%(prop, ','.join([a]*len(values)))) - tables = [] - if where: - tables.append('select id as nodeid from _%s where %s'%( - self.classname, ' and '.join(where))) - - # now multilinks - for prop, values in propspec: - if not isinstance(props[prop], hyperdb.Multilink): - continue - if type(values) is type(''): - allvalues += (values,) - s = a - else: - allvalues += tuple(values.keys()) - s = ','.join([a]*len(values)) - tables.append('select nodeid from %s_%s where linkid in (%s)'%( - self.classname, prop, s)) - sql = '\nunion\n'.join(tables) - self.db.sql(sql, allvalues) - l = [x[0] for x in self.db.sql_fetchall()] - if __debug__: - print >>hyperdb.DEBUG, 'find ... ', l - return l - - def stringFind(self, **requirements): - '''Locate a particular node by matching a set of its String - properties in a caseless search. - - If the property is not a String property, a TypeError is raised. - - The return is a list of the id of all nodes that match. - ''' - where = [] - args = [] - for propname in requirements.keys(): - prop = self.properties[propname] - if isinstance(not prop, String): - raise TypeError, "'%s' not a String property"%propname - where.append(propname) - args.append(requirements[propname].lower()) - - # generate the where clause - s = ' and '.join(['_%s=%s'%(col, self.db.arg) for col in where]) - sql = 'select id from _%s where %s'%(self.classname, s) - self.db.sql(sql, tuple(args)) - l = [x[0] for x in self.db.sql_fetchall()] - if __debug__: - print >>hyperdb.DEBUG, 'find ... ', l - return l - - def list(self): - ''' Return a list of the ids of the active nodes in this class. - ''' - return self.db.getnodeids(self.classname, retired=0) - - def filter(self, search_matches, filterspec, sort, group): - ''' Return a list of the ids of the active nodes in this class that - match the 'filter' spec, sorted by the group spec and then the - sort spec - - "filterspec" is {propname: value(s)} - "sort" and "group" are (dir, prop) where dir is '+', '-' or None - and prop is a prop name or None - "search_matches" is {nodeid: marker} - - The filter must match all properties specificed - but if the - property value to match is a list, any one of the values in the - list may match for that property to match. - ''' - # just don't bother if the full-text search matched diddly - if search_matches == {}: - return [] - - cn = self.classname - - # figure the WHERE clause from the filterspec - props = self.getprops() - frum = ['_'+cn] - where = [] - args = [] - a = self.db.arg - for k, v in filterspec.items(): - propclass = props[k] - # now do other where clause stuff - if isinstance(propclass, Multilink): - tn = '%s_%s'%(cn, k) - frum.append(tn) - if isinstance(v, type([])): - s = ','.join([a for x in v]) - where.append('id=%s.nodeid and %s.linkid in (%s)'%(tn,tn,s)) - args = args + v - else: - where.append('id=%s.nodeid and %s.linkid = %s'%(tn, tn, a)) - args.append(v) - elif isinstance(propclass, String): - if not isinstance(v, type([])): - v = [v] - - # Quote the bits in the string that need it and then embed - # in a "substring" search. Note - need to quote the '%' so - # they make it through the python layer happily - v = ['%%'+self.db.sql_stringquote(s)+'%%' for s in v] - - # now add to the where clause - where.append(' or '.join(["_%s LIKE '%s'"%(k, s) for s in v])) - # note: args are embedded in the query string now - elif isinstance(propclass, Link): - if isinstance(v, type([])): - if '-1' in v: - v.remove('-1') - xtra = ' or _%s is NULL'%k - else: - xtra = '' - s = ','.join([a for x in v]) - where.append('(_%s in (%s)%s)'%(k, s, xtra)) - args = args + v - else: - if v == '-1': - v = None - where.append('_%s is NULL'%k) - else: - where.append('_%s=%s'%(k, a)) - args.append(v) - else: - if isinstance(v, type([])): - s = ','.join([a for x in v]) - where.append('_%s in (%s)'%(k, s)) - args = args + v - else: - where.append('_%s=%s'%(k, a)) - args.append(v) - - # add results of full text search - if search_matches is not None: - v = search_matches.keys() - s = ','.join([a for x in v]) - where.append('id in (%s)'%s) - args = args + v - - # "grouping" is just the first-order sorting in the SQL fetch - # can modify it...) - orderby = [] - ordercols = [] - if group[0] is not None and group[1] is not None: - if group[0] != '-': - orderby.append('_'+group[1]) - ordercols.append('_'+group[1]) - else: - orderby.append('_'+group[1]+' desc') - ordercols.append('_'+group[1]) - - # now add in the sorting - group = '' - if sort[0] is not None and sort[1] is not None: - direction, colname = sort - if direction != '-': - if colname == 'id': - orderby.append(colname) - else: - orderby.append('_'+colname) - ordercols.append('_'+colname) - else: - if colname == 'id': - orderby.append(colname+' desc') - ordercols.append(colname) - else: - orderby.append('_'+colname+' desc') - ordercols.append('_'+colname) - - # construct the SQL - frum = ','.join(frum) - if where: - where = ' where ' + (' and '.join(where)) - else: - where = '' - cols = ['id'] - if orderby: - cols = cols + ordercols - order = ' order by %s'%(','.join(orderby)) - else: - order = '' - cols = ','.join(cols) - sql = 'select %s from %s %s%s%s'%(cols, frum, where, group, order) - args = tuple(args) - if __debug__: - print >>hyperdb.DEBUG, 'filter', (self, sql, args) - self.db.cursor.execute(sql, args) - l = self.db.cursor.fetchall() - - # return the IDs (the first column) - # XXX The filter(None, l) bit is sqlite-specific... if there's _NO_ - # XXX matches to a fetch, it returns NULL instead of nothing!?! - return filter(None, [row[0] for row in l]) - - def count(self): - '''Get the number of nodes in this class. - - If the returned integer is 'numnodes', the ids of all the nodes - in this class run from 1 to numnodes, and numnodes+1 will be the - id of the next node to be created in this class. - ''' - return self.db.countnodes(self.classname) - - # Manipulating properties: - def getprops(self, protected=1): - '''Return a dictionary mapping property names to property objects. - If the "protected" flag is true, we include protected properties - - those which may not be modified. - ''' - d = self.properties.copy() - if protected: - d['id'] = String() - d['creation'] = hyperdb.Date() - d['activity'] = hyperdb.Date() - d['creator'] = hyperdb.Link('user') - return d - - def addprop(self, **properties): - '''Add properties to this class. - - The keyword arguments in 'properties' must map names to property - objects, or a TypeError is raised. None of the keys in 'properties' - may collide with the names of existing properties, or a ValueError - is raised before any properties have been added. - ''' - for key in properties.keys(): - if self.properties.has_key(key): - raise ValueError, key - self.properties.update(properties) - - def index(self, nodeid): - '''Add (or refresh) the node to search indexes - ''' - # find all the String properties that have indexme - for prop, propclass in self.getprops().items(): - if isinstance(propclass, String) and propclass.indexme: - try: - value = str(self.get(nodeid, prop)) - except IndexError: - # node no longer exists - entry should be removed - self.db.indexer.purge_entry((self.classname, nodeid, prop)) - else: - # and index them under (classname, nodeid, property) - self.db.indexer.add_text((self.classname, nodeid, prop), - value) - - - # - # Detector interface - # - def audit(self, event, detector): - '''Register a detector - ''' - l = self.auditors[event] - if detector not in l: - self.auditors[event].append(detector) - - def fireAuditors(self, action, nodeid, newvalues): - '''Fire all registered auditors. - ''' - for audit in self.auditors[action]: - audit(self.db, self, nodeid, newvalues) - - def react(self, event, detector): - '''Register a detector - ''' - l = self.reactors[event] - if detector not in l: - self.reactors[event].append(detector) - - def fireReactors(self, action, nodeid, oldvalues): - '''Fire all registered reactors. - ''' - for react in self.reactors[action]: - react(self.db, self, nodeid, oldvalues) - -class FileClass(Class): - '''This class defines a large chunk of data. To support this, it has a - mandatory String property "content" which is typically saved off - externally to the hyperdb. - - The default MIME type of this data is defined by the - "default_mime_type" class attribute, which may be overridden by each - node if the class defines a "type" String property. - ''' - default_mime_type = 'text/plain' - - def create(self, **propvalues): - ''' snaffle the file propvalue and store in a file - ''' - content = propvalues['content'] - del propvalues['content'] - newid = Class.create(self, **propvalues) - self.db.storefile(self.classname, newid, None, content) - return newid - - def import_list(self, propnames, proplist): - ''' Trap the "content" property... - ''' - # dupe this list so we don't affect others - propnames = propnames[:] - - # extract the "content" property from the proplist - i = propnames.index('content') - content = eval(proplist[i]) - del propnames[i] - del proplist[i] - - # do the normal import - newid = Class.import_list(self, propnames, proplist) - - # save off the "content" file - self.db.storefile(self.classname, newid, None, content) - return newid - - _marker = [] - def get(self, nodeid, propname, default=_marker, cache=1): - ''' trap the content propname and get it from the file - ''' - - poss_msg = 'Possibly a access right configuration problem.' - if propname == 'content': - try: - return self.db.getfile(self.classname, nodeid, None) - except IOError, (strerror): - # BUG: by catching this we donot see an error in the log. - return 'ERROR reading file: %s%s\n%s\n%s'%( - self.classname, nodeid, poss_msg, strerror) - if default is not self._marker: - return Class.get(self, nodeid, propname, default, cache=cache) - else: - return Class.get(self, nodeid, propname, cache=cache) - - def getprops(self, protected=1): - ''' In addition to the actual properties on the node, these methods - provide the "content" property. If the "protected" flag is true, - we include protected properties - those which may not be - modified. - ''' - d = Class.getprops(self, protected=protected).copy() - d['content'] = hyperdb.String() - return d - - def index(self, nodeid): - ''' Index the node in the search index. - - We want to index the content in addition to the normal String - property indexing. - ''' - # perform normal indexing - Class.index(self, nodeid) - - # get the content to index - content = self.get(nodeid, 'content') - - # figure the mime type - if self.properties.has_key('type'): - mime_type = self.get(nodeid, 'type') - else: - mime_type = self.default_mime_type - - # and index! - self.db.indexer.add_text((self.classname, nodeid, 'content'), content, - mime_type) - -# XXX deviation from spec - was called ItemClass -class IssueClass(Class, roundupdb.IssueClass): - # Overridden methods: - def __init__(self, db, classname, **properties): - '''The newly-created class automatically includes the "messages", - "files", "nosy", and "superseder" properties. If the 'properties' - dictionary attempts to specify any of these properties or a - "creation" or "activity" property, a ValueError is raised. - ''' - if not properties.has_key('title'): - properties['title'] = hyperdb.String(indexme='yes') - if not properties.has_key('messages'): - properties['messages'] = hyperdb.Multilink("msg") - if not properties.has_key('files'): - properties['files'] = hyperdb.Multilink("file") - if not properties.has_key('nosy'): - # note: journalling is turned off as it really just wastes - # space. this behaviour may be overridden in an instance - properties['nosy'] = hyperdb.Multilink("user", do_journal="no") - if not properties.has_key('superseder'): - properties['superseder'] = hyperdb.Multilink(classname) - Class.__init__(self, db, classname, **properties) -
--- a/roundup/cgi/PageTemplates/PythonExpr.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,87 +0,0 @@ -############################################################################## -# -# Copyright (c) 2001 Zope Corporation and Contributors. All Rights Reserved. -# -# This software is subject to the provisions of the Zope Public License, -# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution. -# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED -# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS -# FOR A PARTICULAR PURPOSE -# -############################################################################## - -"""Generic Python Expression Handler - -Modified for Roundup 0.5 release: - -- more informative traceback info - -""" - -__version__='$Revision: 1.3 $'[11:-2] - -from TALES import CompilerError -from string import strip, split, join, replace, lstrip -from sys import exc_info - -class getSecurityManager: - '''Null security manager''' - def validate(self, *args, **kwargs): - return 1 - addContext = removeContext = validateValue = validate - -class PythonExpr: - def __init__(self, name, expr, engine): - self.expr = expr = replace(strip(expr), '\n', ' ') - try: - d = {} - exec 'def f():\n return %s\n' % strip(expr) in d - self._f = d['f'] - except: - raise CompilerError, ('Python expression error:\n' - '%s: %s') % exc_info()[:2] - self._get_used_names() - - def _get_used_names(self): - self._f_varnames = vnames = [] - for vname in self._f.func_code.co_names: - if vname[0] not in '$_': - vnames.append(vname) - - def _bind_used_names(self, econtext): - # Bind template variables - names = {} - vars = econtext.vars - getType = econtext._engine.getTypes().get - for vname in self._f_varnames: - has, val = vars.has_get(vname) - if not has: - has = val = getType(vname) - if has: - val = ExprTypeProxy(vname, val, econtext) - if has: - names[vname] = val - return names - - def __call__(self, econtext): - __traceback_info__ = 'python expression "%s"'%self.expr - f = self._f - f.func_globals.update(self._bind_used_names(econtext)) - return f() - - def __str__(self): - return 'Python expression "%s"' % self.expr - def __repr__(self): - return '<PythonExpr %s>' % self.expr - -class ExprTypeProxy: - '''Class that proxies access to an expression type handler''' - def __init__(self, name, handler, econtext): - self._name = name - self._handler = handler - self._econtext = econtext - def __call__(self, text): - return self._handler(self._name, text, - self._econtext._engine)(self._econtext) -
--- a/roundup/cgi/PageTemplates/TALES.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,288 +0,0 @@ -############################################################################## -# -# Copyright (c) 2001 Zope Corporation and Contributors. All Rights Reserved. -# -# This software is subject to the provisions of the Zope Public License, -# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution. -# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED -# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS -# FOR A PARTICULAR PURPOSE -# -############################################################################## -"""TALES - -An implementation of a generic TALES engine - -Modified for Roundup 0.5 release: - -- changed imports to import from roundup.cgi -""" - -__version__='$Revision: 1.3 $'[11:-2] - -import re, sys -from roundup.cgi import ZTUtils -from MultiMapping import MultiMapping - -StringType = type('') - -NAME_RE = r"[a-zA-Z][a-zA-Z0-9_]*" -_parse_expr = re.compile(r"(%s):" % NAME_RE).match -_valid_name = re.compile('%s$' % NAME_RE).match - -class TALESError(Exception): - """Error during TALES expression evaluation""" - -class Undefined(TALESError): - '''Exception raised on traversal of an undefined path''' - -class RegistrationError(Exception): - '''TALES Type Registration Error''' - -class CompilerError(Exception): - '''TALES Compiler Error''' - -class Default: - '''Retain Default''' -Default = Default() - -_marker = [] - -class SafeMapping(MultiMapping): - '''Mapping with security declarations and limited method exposure. - - Since it subclasses MultiMapping, this class can be used to wrap - one or more mapping objects. Restricted Python code will not be - able to mutate the SafeMapping or the wrapped mappings, but will be - able to read any value. - ''' - __allow_access_to_unprotected_subobjects__ = 1 - push = pop = None - - _push = MultiMapping.push - _pop = MultiMapping.pop - - def has_get(self, key, _marker=[]): - v = self.get(key, _marker) - return v is not _marker, v - -class Iterator(ZTUtils.Iterator): - def __init__(self, name, seq, context): - ZTUtils.Iterator.__init__(self, seq) - self.name = name - self._context = context - - def next(self): - if ZTUtils.Iterator.next(self): - self._context.setLocal(self.name, self.item) - return 1 - return 0 - - -class ErrorInfo: - """Information about an exception passed to an on-error handler.""" - __allow_access_to_unprotected_subobjects__ = 1 - - def __init__(self, err, position=(None, None)): - if isinstance(err, Exception): - self.type = err.__class__ - self.value = err - else: - self.type = err - self.value = None - self.lineno = position[0] - self.offset = position[1] - - -class Engine: - '''Expression Engine - - An instance of this class keeps a mutable collection of expression - type handlers. It can compile expression strings by delegating to - these handlers. It can provide an expression Context, which is - capable of holding state and evaluating compiled expressions. - ''' - Iterator = Iterator - - def __init__(self, Iterator=None): - self.types = {} - if Iterator is not None: - self.Iterator = Iterator - - def registerType(self, name, handler): - if not _valid_name(name): - raise RegistrationError, 'Invalid Expression type "%s".' % name - types = self.types - if types.has_key(name): - raise RegistrationError, ( - 'Multiple registrations for Expression type "%s".' % - name) - types[name] = handler - - def getTypes(self): - return self.types - - def compile(self, expression): - m = _parse_expr(expression) - if m: - type = m.group(1) - expr = expression[m.end():] - else: - type = "standard" - expr = expression - try: - handler = self.types[type] - except KeyError: - raise CompilerError, ( - 'Unrecognized expression type "%s".' % type) - return handler(type, expr, self) - - def getContext(self, contexts=None, **kwcontexts): - if contexts is not None: - if kwcontexts: - kwcontexts.update(contexts) - else: - kwcontexts = contexts - return Context(self, kwcontexts) - - def getCompilerError(self): - return CompilerError - -class Context: - '''Expression Context - - An instance of this class holds context information that it can - use to evaluate compiled expressions. - ''' - - _context_class = SafeMapping - position = (None, None) - source_file = None - - def __init__(self, engine, contexts): - self._engine = engine - self.contexts = contexts - contexts['nothing'] = None - contexts['default'] = Default - - self.repeat_vars = rv = {} - # Wrap this, as it is visible to restricted code - contexts['repeat'] = rep = self._context_class(rv) - contexts['loop'] = rep # alias - - self.global_vars = gv = contexts.copy() - self.local_vars = lv = {} - self.vars = self._context_class(gv, lv) - - # Keep track of what needs to be popped as each scope ends. - self._scope_stack = [] - - def beginScope(self): - self._scope_stack.append([self.local_vars.copy()]) - - def endScope(self): - scope = self._scope_stack.pop() - self.local_vars = lv = scope[0] - v = self.vars - v._pop() - v._push(lv) - # Pop repeat variables, if any - i = len(scope) - 1 - while i: - name, value = scope[i] - if value is None: - del self.repeat_vars[name] - else: - self.repeat_vars[name] = value - i = i - 1 - - def setLocal(self, name, value): - self.local_vars[name] = value - - def setGlobal(self, name, value): - self.global_vars[name] = value - - def setRepeat(self, name, expr): - expr = self.evaluate(expr) - if not expr: - return self._engine.Iterator(name, (), self) - it = self._engine.Iterator(name, expr, self) - old_value = self.repeat_vars.get(name) - self._scope_stack[-1].append((name, old_value)) - self.repeat_vars[name] = it - return it - - def evaluate(self, expression, - isinstance=isinstance, StringType=StringType): - if isinstance(expression, StringType): - expression = self._engine.compile(expression) - __traceback_supplement__ = ( - TALESTracebackSupplement, self, expression) - v = expression(self) - return v - - evaluateValue = evaluate - evaluateBoolean = evaluate - - def evaluateText(self, expr, None=None): - text = self.evaluate(expr) - if text is Default or text is None: - return text - return str(text) - - def evaluateStructure(self, expr): - return self.evaluate(expr) - evaluateStructure = evaluate - - def evaluateMacro(self, expr): - # XXX Should return None or a macro definition - return self.evaluate(expr) - evaluateMacro = evaluate - - def createErrorInfo(self, err, position): - return ErrorInfo(err, position) - - def getDefault(self): - return Default - - def setSourceFile(self, source_file): - self.source_file = source_file - - def setPosition(self, position): - self.position = position - - - -class TALESTracebackSupplement: - """Implementation of ITracebackSupplement""" - def __init__(self, context, expression): - self.context = context - self.source_url = context.source_file - self.line = context.position[0] - self.column = context.position[1] - self.expression = repr(expression) - - def getInfo(self, as_html=0): - import pprint - data = self.context.contexts.copy() - s = pprint.pformat(data) - if not as_html: - return ' - Names:\n %s' % string.replace(s, '\n', '\n ') - else: - from cgi import escape - return '<b>Names:</b><pre>%s</pre>' % (escape(s)) - return None - - - -class SimpleExpr: - '''Simple example of an expression type handler''' - def __init__(self, name, expr, engine): - self._name = name - self._expr = expr - def __call__(self, econtext): - return self._name, self._expr - def __repr__(self): - return '<SimpleExpr %s %s>' % (self._name, `self._expr`) -
--- a/roundup/cgi/TAL/TALGenerator.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,583 +0,0 @@ -############################################################################## -# -# Copyright (c) 2001, 2002 Zope Corporation and Contributors. -# All Rights Reserved. -# -# This software is subject to the provisions of the Zope Public License, -# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution. -# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED -# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS -# FOR A PARTICULAR PURPOSE -# -############################################################################## -""" -Code generator for TALInterpreter intermediate code. -""" - -import string -import re -import cgi - -from TALDefs import * - -class TALGenerator: - - inMacroUse = 0 - inMacroDef = 0 - source_file = None - - def __init__(self, expressionCompiler=None, xml=1, source_file=None): - if not expressionCompiler: - from DummyEngine import DummyEngine - expressionCompiler = DummyEngine() - self.expressionCompiler = expressionCompiler - self.CompilerError = expressionCompiler.getCompilerError() - self.program = [] - self.stack = [] - self.todoStack = [] - self.macros = {} - self.slots = {} - self.slotStack = [] - self.xml = xml - self.emit("version", TAL_VERSION) - self.emit("mode", xml and "xml" or "html") - if source_file is not None: - self.source_file = source_file - self.emit("setSourceFile", source_file) - - def getCode(self): - assert not self.stack - assert not self.todoStack - return self.optimize(self.program), self.macros - - def optimize(self, program): - output = [] - collect = [] - rawseen = cursor = 0 - if self.xml: - endsep = "/>" - else: - endsep = " />" - for cursor in xrange(len(program)+1): - try: - item = program[cursor] - except IndexError: - item = (None, None) - opcode = item[0] - if opcode == "rawtext": - collect.append(item[1]) - continue - if opcode == "endTag": - collect.append("</%s>" % item[1]) - continue - if opcode == "startTag": - if self.optimizeStartTag(collect, item[1], item[2], ">"): - continue - if opcode == "startEndTag": - if self.optimizeStartTag(collect, item[1], item[2], endsep): - continue - if opcode in ("beginScope", "endScope"): - # Push *Scope instructions in front of any text instructions; - # this allows text instructions separated only by *Scope - # instructions to be joined together. - output.append(self.optimizeArgsList(item)) - continue - text = string.join(collect, "") - if text: - i = string.rfind(text, "\n") - if i >= 0: - i = len(text) - (i + 1) - output.append(("rawtextColumn", (text, i))) - else: - output.append(("rawtextOffset", (text, len(text)))) - if opcode != None: - output.append(self.optimizeArgsList(item)) - rawseen = cursor+1 - collect = [] - return self.optimizeCommonTriple(output) - - def optimizeArgsList(self, item): - if len(item) == 2: - return item - else: - return item[0], tuple(item[1:]) - - actionIndex = {"replace":0, "insert":1, "metal":2, "tal":3, "xmlns":4, - 0: 0, 1: 1, 2: 2, 3: 3, 4: 4} - def optimizeStartTag(self, collect, name, attrlist, end): - if not attrlist: - collect.append("<%s%s" % (name, end)) - return 1 - opt = 1 - new = ["<" + name] - for i in range(len(attrlist)): - item = attrlist[i] - if len(item) > 2: - opt = 0 - name, value, action = item[:3] - action = self.actionIndex[action] - attrlist[i] = (name, value, action) + item[3:] - else: - if item[1] is None: - s = item[0] - else: - s = "%s=%s" % (item[0], quote(item[1])) - attrlist[i] = item[0], s - if item[1] is None: - new.append(" " + item[0]) - else: - new.append(" %s=%s" % (item[0], quote(item[1]))) - if opt: - new.append(end) - collect.extend(new) - return opt - - def optimizeCommonTriple(self, program): - if len(program) < 3: - return program - output = program[:2] - prev2, prev1 = output - for item in program[2:]: - if ( item[0] == "beginScope" - and prev1[0] == "setPosition" - and prev2[0] == "rawtextColumn"): - position = output.pop()[1] - text, column = output.pop()[1] - prev1 = None, None - closeprev = 0 - if output and output[-1][0] == "endScope": - closeprev = 1 - output.pop() - item = ("rawtextBeginScope", - (text, column, position, closeprev, item[1])) - output.append(item) - prev2 = prev1 - prev1 = item - return output - - def todoPush(self, todo): - self.todoStack.append(todo) - - def todoPop(self): - return self.todoStack.pop() - - def compileExpression(self, expr): - try: - return self.expressionCompiler.compile(expr) - except self.CompilerError, err: - raise TALError('%s in expression %s' % (err.args[0], `expr`), - self.position) - - def pushProgram(self): - self.stack.append(self.program) - self.program = [] - - def popProgram(self): - program = self.program - self.program = self.stack.pop() - return self.optimize(program) - - def pushSlots(self): - self.slotStack.append(self.slots) - self.slots = {} - - def popSlots(self): - slots = self.slots - self.slots = self.slotStack.pop() - return slots - - def emit(self, *instruction): - self.program.append(instruction) - - def emitStartTag(self, name, attrlist, isend=0): - if isend: - opcode = "startEndTag" - else: - opcode = "startTag" - self.emit(opcode, name, attrlist) - - def emitEndTag(self, name): - if self.xml and self.program and self.program[-1][0] == "startTag": - # Minimize empty element - self.program[-1] = ("startEndTag",) + self.program[-1][1:] - else: - self.emit("endTag", name) - - def emitOptTag(self, name, optTag, isend): - program = self.popProgram() #block - start = self.popProgram() #start tag - if (isend or not program) and self.xml: - # Minimize empty element - start[-1] = ("startEndTag",) + start[-1][1:] - isend = 1 - cexpr = optTag[0] - if cexpr: - cexpr = self.compileExpression(optTag[0]) - self.emit("optTag", name, cexpr, optTag[1], isend, start, program) - - def emitRawText(self, text): - self.emit("rawtext", text) - - def emitText(self, text): - self.emitRawText(cgi.escape(text)) - - def emitDefines(self, defines): - for part in splitParts(defines): - m = re.match( - r"(?s)\s*(?:(global|local)\s+)?(%s)\s+(.*)\Z" % NAME_RE, part) - if not m: - raise TALError("invalid define syntax: " + `part`, - self.position) - scope, name, expr = m.group(1, 2, 3) - scope = scope or "local" - cexpr = self.compileExpression(expr) - if scope == "local": - self.emit("setLocal", name, cexpr) - else: - self.emit("setGlobal", name, cexpr) - - def emitOnError(self, name, onError): - block = self.popProgram() - key, expr = parseSubstitution(onError) - cexpr = self.compileExpression(expr) - if key == "text": - self.emit("insertText", cexpr, []) - else: - assert key == "structure" - self.emit("insertStructure", cexpr, {}, []) - self.emitEndTag(name) - handler = self.popProgram() - self.emit("onError", block, handler) - - def emitCondition(self, expr): - cexpr = self.compileExpression(expr) - program = self.popProgram() - self.emit("condition", cexpr, program) - - def emitRepeat(self, arg): - m = re.match("(?s)\s*(%s)\s+(.*)\Z" % NAME_RE, arg) - if not m: - raise TALError("invalid repeat syntax: " + `arg`, - self.position) - name, expr = m.group(1, 2) - cexpr = self.compileExpression(expr) - program = self.popProgram() - self.emit("loop", name, cexpr, program) - - def emitSubstitution(self, arg, attrDict={}): - key, expr = parseSubstitution(arg) - cexpr = self.compileExpression(expr) - program = self.popProgram() - if key == "text": - self.emit("insertText", cexpr, program) - else: - assert key == "structure" - self.emit("insertStructure", cexpr, attrDict, program) - - def emitDefineMacro(self, macroName): - program = self.popProgram() - macroName = string.strip(macroName) - if self.macros.has_key(macroName): - raise METALError("duplicate macro definition: %s" % `macroName`, - self.position) - if not re.match('%s$' % NAME_RE, macroName): - raise METALError("invalid macro name: %s" % `macroName`, - self.position) - self.macros[macroName] = program - self.inMacroDef = self.inMacroDef - 1 - self.emit("defineMacro", macroName, program) - - def emitUseMacro(self, expr): - cexpr = self.compileExpression(expr) - program = self.popProgram() - self.inMacroUse = 0 - self.emit("useMacro", expr, cexpr, self.popSlots(), program) - - def emitDefineSlot(self, slotName): - program = self.popProgram() - slotName = string.strip(slotName) - if not re.match('%s$' % NAME_RE, slotName): - raise METALError("invalid slot name: %s" % `slotName`, - self.position) - self.emit("defineSlot", slotName, program) - - def emitFillSlot(self, slotName): - program = self.popProgram() - slotName = string.strip(slotName) - if self.slots.has_key(slotName): - raise METALError("duplicate fill-slot name: %s" % `slotName`, - self.position) - if not re.match('%s$' % NAME_RE, slotName): - raise METALError("invalid slot name: %s" % `slotName`, - self.position) - self.slots[slotName] = program - self.inMacroUse = 1 - self.emit("fillSlot", slotName, program) - - def unEmitWhitespace(self): - collect = [] - i = len(self.program) - 1 - while i >= 0: - item = self.program[i] - if item[0] != "rawtext": - break - text = item[1] - if not re.match(r"\A\s*\Z", text): - break - collect.append(text) - i = i-1 - del self.program[i+1:] - if i >= 0 and self.program[i][0] == "rawtext": - text = self.program[i][1] - m = re.search(r"\s+\Z", text) - if m: - self.program[i] = ("rawtext", text[:m.start()]) - collect.append(m.group()) - collect.reverse() - return string.join(collect, "") - - def unEmitNewlineWhitespace(self): - collect = [] - i = len(self.program) - while i > 0: - i = i-1 - item = self.program[i] - if item[0] != "rawtext": - break - text = item[1] - if re.match(r"\A[ \t]*\Z", text): - collect.append(text) - continue - m = re.match(r"(?s)^(.*)(\n[ \t]*)\Z", text) - if not m: - break - text, rest = m.group(1, 2) - collect.reverse() - rest = rest + string.join(collect, "") - del self.program[i:] - if text: - self.emit("rawtext", text) - return rest - return None - - def replaceAttrs(self, attrlist, repldict): - if not repldict: - return attrlist - newlist = [] - for item in attrlist: - key = item[0] - if repldict.has_key(key): - item = item[:2] + ("replace", repldict[key]) - del repldict[key] - newlist.append(item) - for key, value in repldict.items(): # Add dynamic-only attributes - item = (key, None, "insert", value) - newlist.append(item) - return newlist - - def emitStartElement(self, name, attrlist, taldict, metaldict, - position=(None, None), isend=0): - if not taldict and not metaldict: - # Handle the simple, common case - self.emitStartTag(name, attrlist, isend) - self.todoPush({}) - if isend: - self.emitEndElement(name, isend) - return - - self.position = position - for key, value in taldict.items(): - if key not in KNOWN_TAL_ATTRIBUTES: - raise TALError("bad TAL attribute: " + `key`, position) - if not (value or key == 'omit-tag'): - raise TALError("missing value for TAL attribute: " + - `key`, position) - for key, value in metaldict.items(): - if key not in KNOWN_METAL_ATTRIBUTES: - raise METALError("bad METAL attribute: " + `key`, - position) - if not value: - raise TALError("missing value for METAL attribute: " + - `key`, position) - todo = {} - defineMacro = metaldict.get("define-macro") - useMacro = metaldict.get("use-macro") - defineSlot = metaldict.get("define-slot") - fillSlot = metaldict.get("fill-slot") - define = taldict.get("define") - condition = taldict.get("condition") - repeat = taldict.get("repeat") - content = taldict.get("content") - replace = taldict.get("replace") - attrsubst = taldict.get("attributes") - onError = taldict.get("on-error") - omitTag = taldict.get("omit-tag") - TALtag = taldict.get("tal tag") - if len(metaldict) > 1 and (defineMacro or useMacro): - raise METALError("define-macro and use-macro cannot be used " - "together or with define-slot or fill-slot", - position) - if content and replace: - raise TALError("content and replace are mutually exclusive", - position) - - repeatWhitespace = None - if repeat: - # Hack to include preceding whitespace in the loop program - repeatWhitespace = self.unEmitNewlineWhitespace() - if position != (None, None): - # XXX at some point we should insist on a non-trivial position - self.emit("setPosition", position) - if self.inMacroUse: - if fillSlot: - self.pushProgram() - if self.source_file is not None: - self.emit("setSourceFile", self.source_file) - todo["fillSlot"] = fillSlot - self.inMacroUse = 0 - else: - if fillSlot: - raise METALError, ("fill-slot must be within a use-macro", - position) - if not self.inMacroUse: - if defineMacro: - self.pushProgram() - self.emit("version", TAL_VERSION) - self.emit("mode", self.xml and "xml" or "html") - if self.source_file is not None: - self.emit("setSourceFile", self.source_file) - todo["defineMacro"] = defineMacro - self.inMacroDef = self.inMacroDef + 1 - if useMacro: - self.pushSlots() - self.pushProgram() - todo["useMacro"] = useMacro - self.inMacroUse = 1 - if defineSlot: - if not self.inMacroDef: - raise METALError, ( - "define-slot must be within a define-macro", - position) - self.pushProgram() - todo["defineSlot"] = defineSlot - - if taldict: - dict = {} - for item in attrlist: - key, value = item[:2] - dict[key] = value - self.emit("beginScope", dict) - todo["scope"] = 1 - if onError: - self.pushProgram() # handler - self.emitStartTag(name, list(attrlist)) # Must copy attrlist! - self.pushProgram() # block - todo["onError"] = onError - if define: - self.emitDefines(define) - todo["define"] = define - if condition: - self.pushProgram() - todo["condition"] = condition - if repeat: - todo["repeat"] = repeat - self.pushProgram() - if repeatWhitespace: - self.emitText(repeatWhitespace) - if content: - todo["content"] = content - if replace: - todo["replace"] = replace - self.pushProgram() - optTag = omitTag is not None or TALtag - if optTag: - todo["optional tag"] = omitTag, TALtag - self.pushProgram() - if attrsubst: - repldict = parseAttributeReplacements(attrsubst) - for key, value in repldict.items(): - repldict[key] = self.compileExpression(value) - else: - repldict = {} - if replace: - todo["repldict"] = repldict - repldict = {} - self.emitStartTag(name, self.replaceAttrs(attrlist, repldict), isend) - if optTag: - self.pushProgram() - if content: - self.pushProgram() - if todo and position != (None, None): - todo["position"] = position - self.todoPush(todo) - if isend: - self.emitEndElement(name, isend) - - def emitEndElement(self, name, isend=0, implied=0): - todo = self.todoPop() - if not todo: - # Shortcut - if not isend: - self.emitEndTag(name) - return - - self.position = position = todo.get("position", (None, None)) - defineMacro = todo.get("defineMacro") - useMacro = todo.get("useMacro") - defineSlot = todo.get("defineSlot") - fillSlot = todo.get("fillSlot") - repeat = todo.get("repeat") - content = todo.get("content") - replace = todo.get("replace") - condition = todo.get("condition") - onError = todo.get("onError") - define = todo.get("define") - repldict = todo.get("repldict", {}) - scope = todo.get("scope") - optTag = todo.get("optional tag") - - if implied > 0: - if defineMacro or useMacro or defineSlot or fillSlot: - exc = METALError - what = "METAL" - else: - exc = TALError - what = "TAL" - raise exc("%s attributes on <%s> require explicit </%s>" % - (what, name, name), position) - - if content: - self.emitSubstitution(content, {}) - if optTag: - self.emitOptTag(name, optTag, isend) - elif not isend: - self.emitEndTag(name) - if replace: - self.emitSubstitution(replace, repldict) - if repeat: - self.emitRepeat(repeat) - if condition: - self.emitCondition(condition) - if onError: - self.emitOnError(name, onError) - if scope: - self.emit("endScope") - if defineSlot: - self.emitDefineSlot(defineSlot) - if fillSlot: - self.emitFillSlot(fillSlot) - if useMacro: - self.emitUseMacro(useMacro) - if defineMacro: - self.emitDefineMacro(defineMacro) - -def test(): - t = TALGenerator() - t.pushProgram() - t.emit("bar") - p = t.popProgram() - t.emit("foo", p) - -if __name__ == "__main__": - test()
--- a/roundup/cgi/TAL/TALInterpreter.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,626 +0,0 @@ -############################################################################## -# -# Copyright (c) 2001, 2002 Zope Corporation and Contributors. -# All Rights Reserved. -# -# This software is subject to the provisions of the Zope Public License, -# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution. -# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED -# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS -# FOR A PARTICULAR PURPOSE -# -############################################################################## -""" -Interpreter for a pre-compiled TAL program. -""" - -import sys -import getopt - -from cgi import escape -from string import join, lower, rfind -try: - from strop import lower, rfind -except ImportError: - pass - -try: - from cStringIO import StringIO -except ImportError: - from StringIO import StringIO - -from TALDefs import quote, TAL_VERSION, TALError, METALError -from TALDefs import isCurrentVersion, getProgramVersion, getProgramMode -from TALGenerator import TALGenerator - -BOOLEAN_HTML_ATTRS = [ - # List of Boolean attributes in HTML that should be rendered in - # minimized form (e.g. <img ismap> rather than <img ismap="">) - # From http://www.w3.org/TR/xhtml1/#guidelines (C.10) - # XXX The problem with this is that this is not valid XML and - # can't be parsed back! - "compact", "nowrap", "ismap", "declare", "noshade", "checked", - "disabled", "readonly", "multiple", "selected", "noresize", - "defer" -] - -EMPTY_HTML_TAGS = [ - # List of HTML tags with an empty content model; these are - # rendered in minimized form, e.g. <img />. - # From http://www.w3.org/TR/xhtml1/#dtds - "base", "meta", "link", "hr", "br", "param", "img", "area", - "input", "col", "basefont", "isindex", "frame", -] - -class AltTALGenerator(TALGenerator): - - def __init__(self, repldict, expressionCompiler=None, xml=0): - self.repldict = repldict - self.enabled = 1 - TALGenerator.__init__(self, expressionCompiler, xml) - - def enable(self, enabled): - self.enabled = enabled - - def emit(self, *args): - if self.enabled: - apply(TALGenerator.emit, (self,) + args) - - def emitStartElement(self, name, attrlist, taldict, metaldict, - position=(None, None), isend=0): - metaldict = {} - taldict = {} - if self.enabled and self.repldict: - taldict["attributes"] = "x x" - TALGenerator.emitStartElement(self, name, attrlist, - taldict, metaldict, position, isend) - - def replaceAttrs(self, attrlist, repldict): - if self.enabled and self.repldict: - repldict = self.repldict - self.repldict = None - return TALGenerator.replaceAttrs(self, attrlist, repldict) - - -class TALInterpreter: - - def __init__(self, program, macros, engine, stream=None, - debug=0, wrap=60, metal=1, tal=1, showtal=-1, - strictinsert=1, stackLimit=100): - self.program = program - self.macros = macros - self.engine = engine - self.Default = engine.getDefault() - self.stream = stream or sys.stdout - self._stream_write = self.stream.write - self.debug = debug - self.wrap = wrap - self.metal = metal - self.tal = tal - if tal: - self.dispatch = self.bytecode_handlers_tal - else: - self.dispatch = self.bytecode_handlers - assert showtal in (-1, 0, 1) - if showtal == -1: - showtal = (not tal) - self.showtal = showtal - self.strictinsert = strictinsert - self.stackLimit = stackLimit - self.html = 0 - self.endsep = "/>" - self.endlen = len(self.endsep) - self.macroStack = [] - self.popMacro = self.macroStack.pop - self.position = None, None # (lineno, offset) - self.col = 0 - self.level = 0 - self.scopeLevel = 0 - self.sourceFile = None - - def saveState(self): - return (self.position, self.col, self.stream, - self.scopeLevel, self.level) - - def restoreState(self, state): - (self.position, self.col, self.stream, scopeLevel, level) = state - self._stream_write = self.stream.write - assert self.level == level - while self.scopeLevel > scopeLevel: - self.engine.endScope() - self.scopeLevel = self.scopeLevel - 1 - self.engine.setPosition(self.position) - - def restoreOutputState(self, state): - (dummy, self.col, self.stream, scopeLevel, level) = state - self._stream_write = self.stream.write - assert self.level == level - assert self.scopeLevel == scopeLevel - - def pushMacro(self, macroName, slots, entering=1): - if len(self.macroStack) >= self.stackLimit: - raise METALError("macro nesting limit (%d) exceeded " - "by %s" % (self.stackLimit, `macroName`)) - self.macroStack.append([macroName, slots, entering]) - - def macroContext(self, what): - macroStack = self.macroStack - i = len(macroStack) - while i > 0: - i = i-1 - if macroStack[i][0] == what: - return i - return -1 - - def __call__(self): - assert self.level == 0 - assert self.scopeLevel == 0 - self.interpret(self.program) - assert self.level == 0 - assert self.scopeLevel == 0 - if self.col > 0: - self._stream_write("\n") - self.col = 0 - - def stream_write(self, s, - len=len, rfind=rfind): - self._stream_write(s) - i = rfind(s, '\n') - if i < 0: - self.col = self.col + len(s) - else: - self.col = len(s) - (i + 1) - - bytecode_handlers = {} - - def interpret(self, program, None=None): - oldlevel = self.level - self.level = oldlevel + 1 - handlers = self.dispatch - try: - if self.debug: - for (opcode, args) in program: - s = "%sdo_%s%s\n" % (" "*self.level, opcode, - repr(args)) - if len(s) > 80: - s = s[:76] + "...\n" - sys.stderr.write(s) - handlers[opcode](self, args) - else: - for (opcode, args) in program: - handlers[opcode](self, args) - finally: - self.level = oldlevel - - def do_version(self, version): - assert version == TAL_VERSION - bytecode_handlers["version"] = do_version - - def do_mode(self, mode): - assert mode in ("html", "xml") - self.html = (mode == "html") - if self.html: - self.endsep = " />" - else: - self.endsep = "/>" - self.endlen = len(self.endsep) - bytecode_handlers["mode"] = do_mode - - def do_setSourceFile(self, source_file): - self.sourceFile = source_file - self.engine.setSourceFile(source_file) - bytecode_handlers["setSourceFile"] = do_setSourceFile - - def do_setPosition(self, position): - self.position = position - self.engine.setPosition(position) - bytecode_handlers["setPosition"] = do_setPosition - - def do_startEndTag(self, stuff): - self.do_startTag(stuff, self.endsep, self.endlen) - bytecode_handlers["startEndTag"] = do_startEndTag - - def do_startTag(self, (name, attrList), - end=">", endlen=1, _len=len): - # The bytecode generator does not cause calls to this method - # for start tags with no attributes; those are optimized down - # to rawtext events. Hence, there is no special "fast path" - # for that case. - _stream_write = self._stream_write - _stream_write("<" + name) - namelen = _len(name) - col = self.col + namelen + 1 - wrap = self.wrap - align = col + 1 - if align >= wrap/2: - align = 4 # Avoid a narrow column far to the right - attrAction = self.dispatch["<attrAction>"] - try: - for item in attrList: - if _len(item) == 2: - name, s = item - else: - ok, name, s = attrAction(self, item) - if not ok: - continue - slen = _len(s) - if (wrap and - col >= align and - col + 1 + slen > wrap): - _stream_write("\n" + " "*align) - col = align + slen - else: - s = " " + s - col = col + 1 + slen - _stream_write(s) - _stream_write(end) - col = col + endlen - finally: - self.col = col - bytecode_handlers["startTag"] = do_startTag - - def attrAction(self, item): - name, value, action = item[:3] - if action == 1 or (action > 1 and not self.showtal): - return 0, name, value - macs = self.macroStack - if action == 2 and self.metal and macs: - if len(macs) > 1 or not macs[-1][2]: - # Drop all METAL attributes at a use-depth above one. - return 0, name, value - # Clear 'entering' flag - macs[-1][2] = 0 - # Convert or drop depth-one METAL attributes. - i = rfind(name, ":") + 1 - prefix, suffix = name[:i], name[i:] - if suffix == "define-macro": - # Convert define-macro as we enter depth one. - name = prefix + "use-macro" - value = macs[-1][0] # Macro name - elif suffix == "define-slot": - name = prefix + "slot" - elif suffix == "fill-slot": - pass - else: - return 0, name, value - - if value is None: - value = name - else: - value = "%s=%s" % (name, quote(value)) - return 1, name, value - - def attrAction_tal(self, item): - name, value, action = item[:3] - if action > 1: - return self.attrAction(item) - ok = 1 - if self.html and lower(name) in BOOLEAN_HTML_ATTRS: - evalue = self.engine.evaluateBoolean(item[3]) - if evalue is self.Default: - if action == 1: # Cancelled insert - ok = 0 - elif evalue: - value = None - else: - ok = 0 - else: - evalue = self.engine.evaluateText(item[3]) - if evalue is self.Default: - if action == 1: # Cancelled insert - ok = 0 - else: - if evalue is None: - ok = 0 - value = evalue - if ok: - if value is None: - value = name - value = "%s=%s" % (name, quote(value)) - return ok, name, value - - bytecode_handlers["<attrAction>"] = attrAction - - def no_tag(self, start, program): - state = self.saveState() - self.stream = stream = StringIO() - self._stream_write = stream.write - self.interpret(start) - self.restoreOutputState(state) - self.interpret(program) - - def do_optTag(self, (name, cexpr, tag_ns, isend, start, program), - omit=0): - if tag_ns and not self.showtal: - return self.no_tag(start, program) - - self.interpret(start) - if not isend: - self.interpret(program) - s = '</%s>' % name - self._stream_write(s) - self.col = self.col + len(s) - - def do_optTag_tal(self, stuff): - cexpr = stuff[1] - if cexpr is not None and (cexpr == '' or - self.engine.evaluateBoolean(cexpr)): - self.no_tag(stuff[-2], stuff[-1]) - else: - self.do_optTag(stuff) - bytecode_handlers["optTag"] = do_optTag - - def dumpMacroStack(self, prefix, suffix, value): - sys.stderr.write("+---- %s%s = %s\n" % (prefix, suffix, value)) - for i in range(len(self.macroStack)): - what, macroName, slots = self.macroStack[i] - sys.stderr.write("| %2d. %-12s %-12s %s\n" % - (i, what, macroName, slots and slots.keys())) - sys.stderr.write("+--------------------------------------\n") - - def do_rawtextBeginScope(self, (s, col, position, closeprev, dict)): - self._stream_write(s) - self.col = col - self.do_setPosition(position) - if closeprev: - engine = self.engine - engine.endScope() - engine.beginScope() - else: - self.engine.beginScope() - self.scopeLevel = self.scopeLevel + 1 - - def do_rawtextBeginScope_tal(self, (s, col, position, closeprev, dict)): - self._stream_write(s) - self.col = col - self.do_setPosition(position) - engine = self.engine - if closeprev: - engine.endScope() - engine.beginScope() - else: - engine.beginScope() - self.scopeLevel = self.scopeLevel + 1 - engine.setLocal("attrs", dict) - bytecode_handlers["rawtextBeginScope"] = do_rawtextBeginScope - - def do_beginScope(self, dict): - self.engine.beginScope() - self.scopeLevel = self.scopeLevel + 1 - - def do_beginScope_tal(self, dict): - engine = self.engine - engine.beginScope() - engine.setLocal("attrs", dict) - self.scopeLevel = self.scopeLevel + 1 - bytecode_handlers["beginScope"] = do_beginScope - - def do_endScope(self, notused=None): - self.engine.endScope() - self.scopeLevel = self.scopeLevel - 1 - bytecode_handlers["endScope"] = do_endScope - - def do_setLocal(self, notused): - pass - - def do_setLocal_tal(self, (name, expr)): - self.engine.setLocal(name, self.engine.evaluateValue(expr)) - bytecode_handlers["setLocal"] = do_setLocal - - def do_setGlobal_tal(self, (name, expr)): - self.engine.setGlobal(name, self.engine.evaluateValue(expr)) - bytecode_handlers["setGlobal"] = do_setLocal - - def do_insertText(self, stuff): - self.interpret(stuff[1]) - - def do_insertText_tal(self, stuff): - text = self.engine.evaluateText(stuff[0]) - if text is None: - return - if text is self.Default: - self.interpret(stuff[1]) - return - s = escape(text) - self._stream_write(s) - i = rfind(s, '\n') - if i < 0: - self.col = self.col + len(s) - else: - self.col = len(s) - (i + 1) - bytecode_handlers["insertText"] = do_insertText - - def do_insertStructure(self, stuff): - self.interpret(stuff[2]) - - def do_insertStructure_tal(self, (expr, repldict, block)): - structure = self.engine.evaluateStructure(expr) - if structure is None: - return - if structure is self.Default: - self.interpret(block) - return - text = str(structure) - if not (repldict or self.strictinsert): - # Take a shortcut, no error checking - self.stream_write(text) - return - if self.html: - self.insertHTMLStructure(text, repldict) - else: - self.insertXMLStructure(text, repldict) - bytecode_handlers["insertStructure"] = do_insertStructure - - def insertHTMLStructure(self, text, repldict): - from HTMLTALParser import HTMLTALParser - gen = AltTALGenerator(repldict, self.engine, 0) - p = HTMLTALParser(gen) # Raises an exception if text is invalid - p.parseString(text) - program, macros = p.getCode() - self.interpret(program) - - def insertXMLStructure(self, text, repldict): - from TALParser import TALParser - gen = AltTALGenerator(repldict, self.engine, 0) - p = TALParser(gen) - gen.enable(0) - p.parseFragment('<!DOCTYPE foo PUBLIC "foo" "bar"><foo>') - gen.enable(1) - p.parseFragment(text) # Raises an exception if text is invalid - gen.enable(0) - p.parseFragment('</foo>', 1) - program, macros = gen.getCode() - self.interpret(program) - - def do_loop(self, (name, expr, block)): - self.interpret(block) - - def do_loop_tal(self, (name, expr, block)): - iterator = self.engine.setRepeat(name, expr) - while iterator.next(): - self.interpret(block) - bytecode_handlers["loop"] = do_loop - - def do_rawtextColumn(self, (s, col)): - self._stream_write(s) - self.col = col - bytecode_handlers["rawtextColumn"] = do_rawtextColumn - - def do_rawtextOffset(self, (s, offset)): - self._stream_write(s) - self.col = self.col + offset - bytecode_handlers["rawtextOffset"] = do_rawtextOffset - - def do_condition(self, (condition, block)): - if not self.tal or self.engine.evaluateBoolean(condition): - self.interpret(block) - bytecode_handlers["condition"] = do_condition - - def do_defineMacro(self, (macroName, macro)): - macs = self.macroStack - if len(macs) == 1: - entering = macs[-1][2] - if not entering: - macs.append(None) - self.interpret(macro) - macs.pop() - return - self.interpret(macro) - bytecode_handlers["defineMacro"] = do_defineMacro - - def do_useMacro(self, (macroName, macroExpr, compiledSlots, block)): - if not self.metal: - self.interpret(block) - return - macro = self.engine.evaluateMacro(macroExpr) - if macro is self.Default: - macro = block - else: - if not isCurrentVersion(macro): - raise METALError("macro %s has incompatible version %s" % - (`macroName`, `getProgramVersion(macro)`), - self.position) - mode = getProgramMode(macro) - if mode != (self.html and "html" or "xml"): - raise METALError("macro %s has incompatible mode %s" % - (`macroName`, `mode`), self.position) - self.pushMacro(macroName, compiledSlots) - saved_source = self.sourceFile - saved_position = self.position # Used by Boa Constructor - self.interpret(macro) - if self.sourceFile != saved_source: - self.engine.setSourceFile(saved_source) - self.sourceFile = saved_source - self.popMacro() - bytecode_handlers["useMacro"] = do_useMacro - - def do_fillSlot(self, (slotName, block)): - # This is only executed if the enclosing 'use-macro' evaluates - # to 'default'. - self.interpret(block) - bytecode_handlers["fillSlot"] = do_fillSlot - - def do_defineSlot(self, (slotName, block)): - if not self.metal: - self.interpret(block) - return - macs = self.macroStack - if macs and macs[-1] is not None: - saved_source = self.sourceFile - saved_position = self.position # Used by Boa Constructor - macroName, slots = self.popMacro()[:2] - slot = slots.get(slotName) - if slot is not None: - self.interpret(slot) - if self.sourceFile != saved_source: - self.engine.setSourceFile(saved_source) - self.sourceFile = saved_source - self.pushMacro(macroName, slots, entering=0) - return - self.pushMacro(macroName, slots) - if len(macs) == 1: - self.interpret(block) - return - self.interpret(block) - bytecode_handlers["defineSlot"] = do_defineSlot - - def do_onError(self, (block, handler)): - self.interpret(block) - - def do_onError_tal(self, (block, handler)): - state = self.saveState() - self.stream = stream = StringIO() - self._stream_write = stream.write - try: - self.interpret(block) - except: - exc = sys.exc_info()[1] - self.restoreState(state) - engine = self.engine - engine.beginScope() - error = engine.createErrorInfo(exc, self.position) - engine.setLocal('error', error) - try: - self.interpret(handler) - finally: - engine.endScope() - else: - self.restoreOutputState(state) - self.stream_write(stream.getvalue()) - bytecode_handlers["onError"] = do_onError - - bytecode_handlers_tal = bytecode_handlers.copy() - bytecode_handlers_tal["rawtextBeginScope"] = do_rawtextBeginScope_tal - bytecode_handlers_tal["beginScope"] = do_beginScope_tal - bytecode_handlers_tal["setLocal"] = do_setLocal_tal - bytecode_handlers_tal["setGlobal"] = do_setGlobal_tal - bytecode_handlers_tal["insertStructure"] = do_insertStructure_tal - bytecode_handlers_tal["insertText"] = do_insertText_tal - bytecode_handlers_tal["loop"] = do_loop_tal - bytecode_handlers_tal["onError"] = do_onError_tal - bytecode_handlers_tal["<attrAction>"] = attrAction_tal - bytecode_handlers_tal["optTag"] = do_optTag_tal - - -def test(): - from driver import FILE, parsefile - from DummyEngine import DummyEngine - try: - opts, args = getopt.getopt(sys.argv[1:], "") - except getopt.error, msg: - print msg - sys.exit(2) - if args: - file = args[0] - else: - file = FILE - doc = parsefile(file) - compiler = TALCompiler(doc) - program, macros = compiler() - engine = DummyEngine() - interpreter = TALInterpreter(program, macros, engine) - interpreter() - -if __name__ == "__main__": - test()
--- a/roundup/cgi/client.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1211 +0,0 @@ -# $Id: client.py,v 1.46 2002-09-26 03:45:09 richard Exp $ - -__doc__ = """ -WWW request handler (also used in the stand-alone server). -""" - -import os, os.path, cgi, StringIO, urlparse, re, traceback, mimetypes, urllib -import binascii, Cookie, time, random - -from roundup import roundupdb, date, hyperdb, password -from roundup.i18n import _ - -from roundup.cgi.templating import Templates, HTMLRequest, NoTemplate -from roundup.cgi import cgitb - -from roundup.cgi.PageTemplates import PageTemplate - -class Unauthorised(ValueError): - pass - -class NotFound(ValueError): - pass - -class Redirect(Exception): - pass - -class SendFile(Exception): - ' Sent a file from the database ' - -class SendStaticFile(Exception): - ' Send a static file from the instance html directory ' - -def initialiseSecurity(security): - ''' Create some Permissions and Roles on the security object - - This function is directly invoked by security.Security.__init__() - as a part of the Security object instantiation. - ''' - security.addPermission(name="Web Registration", - description="User may register through the web") - p = security.addPermission(name="Web Access", - description="User may access the web interface") - security.addPermissionToRole('Admin', p) - - # doing Role stuff through the web - make sure Admin can - p = security.addPermission(name="Web Roles", - description="User may manipulate user Roles through the web") - security.addPermissionToRole('Admin', p) - -class Client: - ''' - A note about login - ------------------ - - If the user has no login cookie, then they are anonymous. There - are two levels of anonymous use. If there is no 'anonymous' user, there - is no login at all and the database is opened in read-only mode. If the - 'anonymous' user exists, the user is logged in using that user (though - there is no cookie). This allows them to modify the database, and all - modifications are attributed to the 'anonymous' user. - - Once a user logs in, they are assigned a session. The Client instance - keeps the nodeid of the session as the "session" attribute. - - Client attributes: - "path" is the PATH_INFO inside the instance (with no leading '/') - "base" is the base URL for the instance - ''' - - def __init__(self, instance, request, env, form=None): - hyperdb.traceMark() - self.instance = instance - self.request = request - self.env = env - - # save off the path - self.path = env['PATH_INFO'] - - # this is the base URL for this instance - self.base = self.instance.config.TRACKER_WEB - - # see if we need to re-parse the environment for the form (eg Zope) - if form is None: - self.form = cgi.FieldStorage(environ=env) - else: - self.form = form - - # turn debugging on/off - try: - self.debug = int(env.get("ROUNDUP_DEBUG", 0)) - except ValueError: - # someone gave us a non-int debug level, turn it off - self.debug = 0 - - # flag to indicate that the HTTP headers have been sent - self.headers_done = 0 - - # additional headers to send with the request - must be registered - # before the first write - self.additional_headers = {} - self.response_code = 200 - - def main(self): - ''' Wrap the real main in a try/finally so we always close off the db. - ''' - try: - self.inner_main() - finally: - if hasattr(self, 'db'): - self.db.close() - - def inner_main(self): - ''' Process a request. - - The most common requests are handled like so: - 1. figure out who we are, defaulting to the "anonymous" user - see determine_user - 2. figure out what the request is for - the context - see determine_context - 3. handle any requested action (item edit, search, ...) - see handle_action - 4. render a template, resulting in HTML output - - In some situations, exceptions occur: - - HTTP Redirect (generally raised by an action) - - SendFile (generally raised by determine_context) - serve up a FileClass "content" property - - SendStaticFile (generally raised by determine_context) - serve up a file from the tracker "html" directory - - Unauthorised (generally raised by an action) - the action is cancelled, the request is rendered and an error - message is displayed indicating that permission was not - granted for the action to take place - - NotFound (raised wherever it needs to be) - percolates up to the CGI interface that called the client - ''' - self.content_action = None - self.ok_message = [] - self.error_message = [] - try: - # make sure we're identified (even anonymously) - self.determine_user() - # figure out the context and desired content template - self.determine_context() - # possibly handle a form submit action (may change self.classname - # and self.template, and may also append error/ok_messages) - self.handle_action() - # now render the page - - # we don't want clients caching our dynamic pages - self.additional_headers['Cache-Control'] = 'no-cache' - self.additional_headers['Pragma'] = 'no-cache' - self.additional_headers['Expires'] = 'Thu, 1 Jan 1970 00:00:00 GMT' - - # render the content - self.write(self.renderContext()) - except Redirect, url: - # let's redirect - if the url isn't None, then we need to do - # the headers, otherwise the headers have been set before the - # exception was raised - if url: - self.additional_headers['Location'] = url - self.response_code = 302 - self.write('Redirecting to <a href="%s">%s</a>'%(url, url)) - except SendFile, designator: - self.serve_file(designator) - except SendStaticFile, file: - self.serve_static_file(str(file)) - except Unauthorised, message: - self.write(self.renderTemplate('page', '', error_message=message)) - except NotFound: - # pass through - raise - except: - # everything else - self.write(cgitb.html()) - - def determine_user(self): - ''' Determine who the user is - ''' - # determine the uid to use - self.opendb('admin') - - # make sure we have the session Class - sessions = self.db.sessions - - # age sessions, remove when they haven't been used for a week - # TODO: this shouldn't be done every access - week = 60*60*24*7 - now = time.time() - for sessid in sessions.list(): - interval = now - sessions.get(sessid, 'last_use') - if interval > week: - sessions.destroy(sessid) - - # look up the user session cookie - cookie = Cookie.Cookie(self.env.get('HTTP_COOKIE', '')) - user = 'anonymous' - - # bump the "revision" of the cookie since the format changed - if (cookie.has_key('roundup_user_2') and - cookie['roundup_user_2'].value != 'deleted'): - - # get the session key from the cookie - self.session = cookie['roundup_user_2'].value - # get the user from the session - try: - # update the lifetime datestamp - sessions.set(self.session, last_use=time.time()) - sessions.commit() - user = sessions.get(self.session, 'user') - except KeyError: - user = 'anonymous' - - # sanity check on the user still being valid, getting the userid - # at the same time - try: - self.userid = self.db.user.lookup(user) - except (KeyError, TypeError): - user = 'anonymous' - - # make sure the anonymous user is valid if we're using it - if user == 'anonymous': - self.make_user_anonymous() - else: - self.user = user - - # reopen the database as the correct user - self.opendb(self.user) - - def determine_context(self, dre=re.compile(r'([^\d]+)(\d+)')): - ''' Determine the context of this page from the URL: - - The URL path after the instance identifier is examined. The path - is generally only one entry long. - - - if there is no path, then we are in the "home" context. - * if the path is "_file", then the additional path entry - specifies the filename of a static file we're to serve up - from the instance "html" directory. Raises a SendStaticFile - exception. - - if there is something in the path (eg "issue"), it identifies - the tracker class we're to display. - - if the path is an item designator (eg "issue123"), then we're - to display a specific item. - * if the path starts with an item designator and is longer than - one entry, then we're assumed to be handling an item of a - FileClass, and the extra path information gives the filename - that the client is going to label the download with (ie - "file123/image.png" is nicer to download than "file123"). This - raises a SendFile exception. - - Both of the "*" types of contexts stop before we bother to - determine the template we're going to use. That's because they - don't actually use templates. - - The template used is specified by the :template CGI variable, - which defaults to: - - only classname suplied: "index" - full item designator supplied: "item" - - We set: - self.classname - the class to display, can be None - self.template - the template to render the current context with - self.nodeid - the nodeid of the class we're displaying - ''' - # default the optional variables - self.classname = None - self.nodeid = None - - # determine the classname and possibly nodeid - path = self.path.split('/') - if not path or path[0] in ('', 'home', 'index'): - if self.form.has_key(':template'): - self.template = self.form[':template'].value - else: - self.template = '' - return - elif path[0] == '_file': - raise SendStaticFile, path[1] - else: - self.classname = path[0] - if len(path) > 1: - # send the file identified by the designator in path[0] - raise SendFile, path[0] - - # see if we got a designator - m = dre.match(self.classname) - if m: - self.classname = m.group(1) - self.nodeid = m.group(2) - if not self.db.getclass(self.classname).hasnode(self.nodeid): - raise NotFound, '%s/%s'%(self.classname, self.nodeid) - # with a designator, we default to item view - self.template = 'item' - else: - # with only a class, we default to index view - self.template = 'index' - - # see if we have a template override - if self.form.has_key(':template'): - self.template = self.form[':template'].value - - # see if we were passed in a message - if self.form.has_key(':ok_message'): - self.ok_message.append(self.form[':ok_message'].value) - if self.form.has_key(':error_message'): - self.error_message.append(self.form[':error_message'].value) - - def serve_file(self, designator, dre=re.compile(r'([^\d]+)(\d+)')): - ''' Serve the file from the content property of the designated item. - ''' - m = dre.match(str(designator)) - if not m: - raise NotFound, str(designator) - classname, nodeid = m.group(1), m.group(2) - if classname != 'file': - raise NotFound, designator - - # we just want to serve up the file named - file = self.db.file - self.additional_headers['Content-Type'] = file.get(nodeid, 'type') - self.write(file.get(nodeid, 'content')) - - def serve_static_file(self, file): - # we just want to serve up the file named - mt = mimetypes.guess_type(str(file))[0] - self.additional_headers['Content-Type'] = mt - self.write(open(os.path.join(self.instance.config.TEMPLATES, - file)).read()) - - def renderContext(self): - ''' Return a PageTemplate for the named page - ''' - name = self.classname - extension = self.template - pt = Templates(self.instance.config.TEMPLATES).get(name, extension) - - # catch errors so we can handle PT rendering errors more nicely - args = { - 'ok_message': self.ok_message, - 'error_message': self.error_message - } - try: - # let the template render figure stuff out - return pt.render(self, None, None, **args) - except NoTemplate, message: - return '<strong>%s</strong>'%message - except: - # everything else - return cgitb.pt_html() - - # these are the actions that are available - actions = ( - ('edit', 'editItemAction'), - ('editCSV', 'editCSVAction'), - ('new', 'newItemAction'), - ('register', 'registerAction'), - ('login', 'loginAction'), - ('logout', 'logout_action'), - ('search', 'searchAction'), - ) - def handle_action(self): - ''' Determine whether there should be an _action called. - - The action is defined by the form variable :action which - identifies the method on this object to call. The four basic - actions are defined in the "actions" sequence on this class: - "edit" -> self.editItemAction - "new" -> self.newItemAction - "register" -> self.registerAction - "login" -> self.loginAction - "logout" -> self.logout_action - "search" -> self.searchAction - - ''' - if not self.form.has_key(':action'): - return None - try: - # get the action, validate it - action = self.form[':action'].value - for name, method in self.actions: - if name == action: - break - else: - raise ValueError, 'No such action "%s"'%action - - # call the mapped action - getattr(self, method)() - except Redirect: - raise - except Unauthorised: - raise - except: - self.db.rollback() - s = StringIO.StringIO() - traceback.print_exc(None, s) - self.error_message.append('<pre>%s</pre>'%cgi.escape(s.getvalue())) - - def write(self, content): - if not self.headers_done: - self.header() - self.request.wfile.write(content) - - def header(self, headers=None, response=None): - '''Put up the appropriate header. - ''' - if headers is None: - headers = {'Content-Type':'text/html'} - if response is None: - response = self.response_code - - # update with additional info - headers.update(self.additional_headers) - - if not headers.has_key('Content-Type'): - headers['Content-Type'] = 'text/html' - self.request.send_response(response) - for entry in headers.items(): - self.request.send_header(*entry) - self.request.end_headers() - self.headers_done = 1 - if self.debug: - self.headers_sent = headers - - def set_cookie(self, user, password): - # TODO generate a much, much stronger session key ;) - self.session = binascii.b2a_base64(repr(random.random())).strip() - - # clean up the base64 - if self.session[-1] == '=': - if self.session[-2] == '=': - self.session = self.session[:-2] - else: - self.session = self.session[:-1] - - # insert the session in the sessiondb - self.db.sessions.set(self.session, user=user, last_use=time.time()) - - # and commit immediately - self.db.sessions.commit() - - # expire us in a long, long time - expire = Cookie._getdate(86400*365) - - # generate the cookie path - make sure it has a trailing '/' - path = '/'.join((self.env['SCRIPT_NAME'], self.env['TRACKER_NAME'], - '')) - self.additional_headers['Set-Cookie'] = \ - 'roundup_user_2=%s; expires=%s; Path=%s;'%(self.session, expire, path) - - def make_user_anonymous(self): - ''' Make us anonymous - - This method used to handle non-existence of the 'anonymous' - user, but that user is mandatory now. - ''' - self.userid = self.db.user.lookup('anonymous') - self.user = 'anonymous' - - def opendb(self, user): - ''' Open the database. - ''' - # open the db if the user has changed - if not hasattr(self, 'db') or user != self.db.journaltag: - if hasattr(self, 'db'): - self.db.close() - self.db = self.instance.open(user) - - # - # Actions - # - def loginAction(self): - ''' Attempt to log a user in. - - Sets up a session for the user which contains the login - credentials. - ''' - # we need the username at a minimum - if not self.form.has_key('__login_name'): - self.error_message.append(_('Username required')) - return - - self.user = self.form['__login_name'].value - # re-open the database for real, using the user - self.opendb(self.user) - if self.form.has_key('__login_password'): - password = self.form['__login_password'].value - else: - password = '' - # make sure the user exists - try: - self.userid = self.db.user.lookup(self.user) - except KeyError: - name = self.user - self.make_user_anonymous() - self.error_message.append(_('No such user "%(name)s"')%locals()) - return - - # and that the password is correct - pw = self.db.user.get(self.userid, 'password') - if password != pw: - self.make_user_anonymous() - self.error_message.append(_('Incorrect password')) - return - - # make sure we're allowed to be here - if not self.loginPermission(): - self.make_user_anonymous() - raise Unauthorised, _("You do not have permission to login") - - # set the session cookie - self.set_cookie(self.user, password) - - def loginPermission(self): - ''' Determine whether the user has permission to log in. - - Base behaviour is to check the user has "Web Access". - ''' - if not self.db.security.hasPermission('Web Access', self.userid): - return 0 - return 1 - - def logout_action(self): - ''' Make us really anonymous - nuke the cookie too - ''' - # log us out - self.make_user_anonymous() - - # construct the logout cookie - now = Cookie._getdate() - path = '/'.join((self.env['SCRIPT_NAME'], self.env['TRACKER_NAME'], - '')) - self.additional_headers['Set-Cookie'] = \ - 'roundup_user_2=deleted; Max-Age=0; expires=%s; Path=%s;'%(now, path) - - # Let the user know what's going on - self.ok_message.append(_('You are logged out')) - - def registerAction(self): - '''Attempt to create a new user based on the contents of the form - and then set the cookie. - - return 1 on successful login - ''' - # create the new user - cl = self.db.user - - # parse the props from the form - try: - props = parsePropsFromForm(self.db, cl, self.form, self.nodeid) - except (ValueError, KeyError), message: - self.error_message.append(_('Error: ') + str(message)) - return - - # make sure we're allowed to register - if not self.registerPermission(props): - raise Unauthorised, _("You do not have permission to register") - - # re-open the database as "admin" - if self.user != 'admin': - self.opendb('admin') - - # create the new user - cl = self.db.user - try: - props = parsePropsFromForm(self.db, cl, self.form) - props['roles'] = self.instance.config.NEW_WEB_USER_ROLES - self.userid = cl.create(**props) - self.db.commit() - except (ValueError, KeyError), message: - self.error_message.append(message) - return - - # log the new user in - self.user = cl.get(self.userid, 'username') - # re-open the database for real, using the user - self.opendb(self.user) - password = self.db.user.get(self.userid, 'password') - self.set_cookie(self.user, password) - - # nice message - message = _('You are now registered, welcome!') - - # redirect to the item's edit page - raise Redirect, '%s%s%s?:ok_message=%s'%( - self.base, self.classname, self.userid, urllib.quote(message)) - - def registerPermission(self, props): - ''' Determine whether the user has permission to register - - Base behaviour is to check the user has "Web Registration". - ''' - # registration isn't allowed to supply roles - if props.has_key('roles'): - return 0 - if self.db.security.hasPermission('Web Registration', self.userid): - return 1 - return 0 - - def editItemAction(self): - ''' Perform an edit of an item in the database. - - Some special form elements: - - :link=designator:property - :multilink=designator:property - The value specifies a node designator and the property on that - node to add _this_ node to as a link or multilink. - :note - Create a message and attach it to the current node's - "messages" property. - :file - Create a file and attach it to the current node's - "files" property. Attach the file to the message created from - the :note if it's supplied. - - :required=property,property,... - The named properties are required to be filled in the form. - - ''' - cl = self.db.classes[self.classname] - - # parse the props from the form - try: - props = parsePropsFromForm(self.db, cl, self.form, self.nodeid) - except (ValueError, KeyError), message: - self.error_message.append(_('Error: ') + str(message)) - return - - # check permission - if not self.editItemPermission(props): - self.error_message.append( - _('You do not have permission to edit %(classname)s'% - self.__dict__)) - return - - # perform the edit - try: - # make changes to the node - props = self._changenode(props) - # handle linked nodes - self._post_editnode(self.nodeid) - except (ValueError, KeyError), message: - self.error_message.append(_('Error: ') + str(message)) - return - - # commit now that all the tricky stuff is done - self.db.commit() - - # and some nice feedback for the user - if props: - message = _('%(changes)s edited ok')%{'changes': - ', '.join(props.keys())} - elif self.form.has_key(':note') and self.form[':note'].value: - message = _('note added') - elif (self.form.has_key(':file') and self.form[':file'].filename): - message = _('file added') - else: - message = _('nothing changed') - - # redirect to the item's edit page - raise Redirect, '%s%s%s?:ok_message=%s'%(self.base, self.classname, - self.nodeid, urllib.quote(message)) - - def editItemPermission(self, props): - ''' Determine whether the user has permission to edit this item. - - Base behaviour is to check the user can edit this class. If we're - editing the "user" class, users are allowed to edit their own - details. Unless it's the "roles" property, which requires the - special Permission "Web Roles". - ''' - # if this is a user node and the user is editing their own node, then - # we're OK - has = self.db.security.hasPermission - if self.classname == 'user': - # reject if someone's trying to edit "roles" and doesn't have the - # right permission. - if props.has_key('roles') and not has('Web Roles', self.userid, - 'user'): - return 0 - # if the item being edited is the current user, we're ok - if self.nodeid == self.userid: - return 1 - if self.db.security.hasPermission('Edit', self.userid, self.classname): - return 1 - return 0 - - def newItemAction(self): - ''' Add a new item to the database. - - This follows the same form as the editItemAction, with the same - special form values. - ''' - cl = self.db.classes[self.classname] - - # parse the props from the form - try: - props = parsePropsFromForm(self.db, cl, self.form, self.nodeid) - except (ValueError, KeyError), message: - self.error_message.append(_('Error: ') + str(message)) - return - - if not self.newItemPermission(props): - self.error_message.append( - _('You do not have permission to create %s' %self.classname)) - - # create a little extra message for anticipated :link / :multilink - if self.form.has_key(':multilink'): - link = self.form[':multilink'].value - elif self.form.has_key(':link'): - link = self.form[':multilink'].value - else: - link = None - xtra = '' - if link: - designator, linkprop = link.split(':') - xtra = ' for <a href="%s">%s</a>'%(designator, designator) - - try: - # do the create - nid = self._createnode(props) - - # handle linked nodes - self._post_editnode(nid) - - # commit now that all the tricky stuff is done - self.db.commit() - - # render the newly created item - self.nodeid = nid - - # and some nice feedback for the user - message = _('%(classname)s created ok')%self.__dict__ + xtra - except (ValueError, KeyError), message: - self.error_message.append(_('Error: ') + str(message)) - return - except: - # oops - self.db.rollback() - s = StringIO.StringIO() - traceback.print_exc(None, s) - self.error_message.append('<pre>%s</pre>'%cgi.escape(s.getvalue())) - return - - # redirect to the new item's page - raise Redirect, '%s%s%s?:ok_message=%s'%(self.base, self.classname, - nid, urllib.quote(message)) - - def newItemPermission(self, props): - ''' Determine whether the user has permission to create (edit) this - item. - - Base behaviour is to check the user can edit this class. No - additional property checks are made. Additionally, new user items - may be created if the user has the "Web Registration" Permission. - ''' - has = self.db.security.hasPermission - if self.classname == 'user' and has('Web Registration', self.userid, - 'user'): - return 1 - if has('Edit', self.userid, self.classname): - return 1 - return 0 - - def editCSVAction(self): - ''' Performs an edit of all of a class' items in one go. - - The "rows" CGI var defines the CSV-formatted entries for the - class. New nodes are identified by the ID 'X' (or any other - non-existent ID) and removed lines are retired. - ''' - # this is per-class only - if not self.editCSVPermission(): - self.error_message.append( - _('You do not have permission to edit %s' %self.classname)) - - # get the CSV module - try: - import csv - except ImportError: - self.error_message.append(_( - 'Sorry, you need the csv module to use this function.<br>\n' - 'Get it from: <a href="http://www.object-craft.com.au/projects/csv/">http://www.object-craft.com.au/projects/csv/')) - return - - cl = self.db.classes[self.classname] - idlessprops = cl.getprops(protected=0).keys() - idlessprops.sort() - props = ['id'] + idlessprops - - # do the edit - rows = self.form['rows'].value.splitlines() - p = csv.parser() - found = {} - line = 0 - for row in rows[1:]: - line += 1 - values = p.parse(row) - # not a complete row, keep going - if not values: continue - - # skip property names header - if values == props: - continue - - # extract the nodeid - nodeid, values = values[0], values[1:] - found[nodeid] = 1 - - # confirm correct weight - if len(idlessprops) != len(values): - self.error_message.append( - _('Not enough values on line %(line)s')%{'line':line}) - return - - # extract the new values - d = {} - for name, value in zip(idlessprops, values): - value = value.strip() - # only add the property if it has a value - if value: - # if it's a multilink, split it - if isinstance(cl.properties[name], hyperdb.Multilink): - value = value.split(':') - d[name] = value - - # perform the edit - if cl.hasnode(nodeid): - # edit existing - cl.set(nodeid, **d) - else: - # new node - found[cl.create(**d)] = 1 - - # retire the removed entries - for nodeid in cl.list(): - if not found.has_key(nodeid): - cl.retire(nodeid) - - # all OK - self.db.commit() - - self.ok_message.append(_('Items edited OK')) - - def editCSVPermission(self): - ''' Determine whether the user has permission to edit this class. - - Base behaviour is to check the user can edit this class. - ''' - if not self.db.security.hasPermission('Edit', self.userid, - self.classname): - return 0 - return 1 - - def searchAction(self): - ''' Mangle some of the form variables. - - Set the form ":filter" variable based on the values of the - filter variables - if they're set to anything other than - "dontcare" then add them to :filter. - - Also handle the ":queryname" variable and save off the query to - the user's query list. - ''' - # generic edit is per-class only - if not self.searchPermission(): - self.error_message.append( - _('You do not have permission to search %s' %self.classname)) - - # add a faked :filter form variable for each filtering prop - props = self.db.classes[self.classname].getprops() - for key in self.form.keys(): - if not props.has_key(key): continue - if not self.form[key].value: continue - self.form.value.append(cgi.MiniFieldStorage(':filter', key)) - - # handle saving the query params - if self.form.has_key(':queryname'): - queryname = self.form[':queryname'].value.strip() - if queryname: - # parse the environment and figure what the query _is_ - req = HTMLRequest(self) - url = req.indexargs_href('', {}) - - # handle editing an existing query - try: - qid = self.db.query.lookup(queryname) - self.db.query.set(qid, klass=self.classname, url=url) - except KeyError: - # create a query - qid = self.db.query.create(name=queryname, - klass=self.classname, url=url) - - # and add it to the user's query multilink - queries = self.db.user.get(self.userid, 'queries') - queries.append(qid) - self.db.user.set(self.userid, queries=queries) - - # commit the query change to the database - self.db.commit() - - def searchPermission(self): - ''' Determine whether the user has permission to search this class. - - Base behaviour is to check the user can view this class. - ''' - if not self.db.security.hasPermission('View', self.userid, - self.classname): - return 0 - return 1 - - def remove_action(self, dre=re.compile(r'([^\d]+)(\d+)')): - # XXX I believe this could be handled by a regular edit action that - # just sets the multilink... - target = self.index_arg(':target')[0] - m = dre.match(target) - if m: - classname = m.group(1) - nodeid = m.group(2) - cl = self.db.getclass(classname) - cl.retire(nodeid) - # now take care of the reference - parentref = self.index_arg(':multilink')[0] - parent, prop = parentref.split(':') - m = dre.match(parent) - if m: - self.classname = m.group(1) - self.nodeid = m.group(2) - cl = self.db.getclass(self.classname) - value = cl.get(self.nodeid, prop) - value.remove(nodeid) - cl.set(self.nodeid, **{prop:value}) - func = getattr(self, 'show%s'%self.classname) - return func() - else: - raise NotFound, parent - else: - raise NotFound, target - - # - # Utility methods for editing - # - def _changenode(self, props): - ''' change the node based on the contents of the form - ''' - cl = self.db.classes[self.classname] - - # create the message - message, files = self._handle_message() - if message: - props['messages'] = cl.get(self.nodeid, 'messages') + [message] - if files: - props['files'] = cl.get(self.nodeid, 'files') + files - - # make the changes - return cl.set(self.nodeid, **props) - - def _createnode(self, props): - ''' create a node based on the contents of the form - ''' - cl = self.db.classes[self.classname] - - # check for messages and files - message, files = self._handle_message() - if message: - props['messages'] = [message] - if files: - props['files'] = files - # create the node and return it's id - return cl.create(**props) - - def _handle_message(self): - ''' generate an edit message - ''' - # handle file attachments - files = [] - if self.form.has_key(':file'): - file = self.form[':file'] - if file.filename: - filename = file.filename.split('\\')[-1] - mime_type = mimetypes.guess_type(filename)[0] - if not mime_type: - mime_type = "application/octet-stream" - # create the new file entry - files.append(self.db.file.create(type=mime_type, - name=filename, content=file.file.read())) - - # we don't want to do a message if none of the following is true... - cn = self.classname - cl = self.db.classes[self.classname] - props = cl.getprops() - note = None - # in a nutshell, don't do anything if there's no note or there's no - # NOSY - if self.form.has_key(':note'): - note = self.form[':note'].value.strip() - if not note: - return None, files - if not props.has_key('messages'): - return None, files - if not isinstance(props['messages'], hyperdb.Multilink): - return None, files - if not props['messages'].classname == 'msg': - return None, files - if not (self.form.has_key('nosy') or note): - return None, files - - # handle the note - if '\n' in note: - summary = re.split(r'\n\r?', note)[0] - else: - summary = note - m = ['%s\n'%note] - - # handle the messageid - # TODO: handle inreplyto - messageid = "<%s.%s.%s@%s>"%(time.time(), random.random(), - self.classname, self.instance.config.MAIL_DOMAIN) - - # now create the message, attaching the files - content = '\n'.join(m) - message_id = self.db.msg.create(author=self.userid, - recipients=[], date=date.Date('.'), summary=summary, - content=content, files=files, messageid=messageid) - - # update the messages property - return message_id, files - - def _post_editnode(self, nid): - '''Do the linking part of the node creation. - - If a form element has :link or :multilink appended to it, its - value specifies a node designator and the property on that node - to add _this_ node to as a link or multilink. - - This is typically used on, eg. the file upload page to indicated - which issue to link the file to. - - TODO: I suspect that this and newfile will go away now that - there's the ability to upload a file using the issue :file form - element! - ''' - cn = self.classname - cl = self.db.classes[cn] - # link if necessary - keys = self.form.keys() - for key in keys: - if key == ':multilink': - value = self.form[key].value - if type(value) != type([]): value = [value] - for value in value: - designator, property = value.split(':') - link, nodeid = hyperdb.splitDesignator(designator) - link = self.db.classes[link] - # take a dupe of the list so we're not changing the cache - value = link.get(nodeid, property)[:] - value.append(nid) - link.set(nodeid, **{property: value}) - elif key == ':link': - value = self.form[key].value - if type(value) != type([]): value = [value] - for value in value: - designator, property = value.split(':') - link, nodeid = hyperdb.splitDesignator(designator) - link = self.db.classes[link] - link.set(nodeid, **{property: nid}) - - -def parsePropsFromForm(db, cl, form, nodeid=0, num_re=re.compile('^\d+$')): - ''' Pull properties for the given class out of the form. - - If a ":required" parameter is supplied, then the names property values - must be supplied or a ValueError will be raised. - ''' - required = [] - if form.has_key(':required'): - value = form[':required'] - if isinstance(value, type([])): - required = [i.value.strip() for i in value] - else: - required = [i.strip() for i in value.value.split(',')] - - props = {} - keys = form.keys() - properties = cl.getprops() - for key in keys: - if not properties.has_key(key): - continue - proptype = properties[key] - - # Get the form value. This value may be a MiniFieldStorage or a list - # of MiniFieldStorages. - value = form[key] - - # make sure non-multilinks only get one value - if not isinstance(proptype, hyperdb.Multilink): - if isinstance(value, type([])): - raise ValueError, 'You have submitted more than one value'\ - ' for the %s property'%key - # we've got a MiniFieldStorage, so pull out the value and strip - # surrounding whitespace - value = value.value.strip() - - if isinstance(proptype, hyperdb.String): - if not value: - continue - elif isinstance(proptype, hyperdb.Password): - if not value: - # ignore empty password values - continue - if not form.has_key('%s:confirm'%key): - raise ValueError, 'Password and confirmation text do not match' - confirm = form['%s:confirm'%key] - if isinstance(confirm, type([])): - raise ValueError, 'You have submitted more than one value'\ - ' for the %s property'%key - if value != confirm.value: - raise ValueError, 'Password and confirmation text do not match' - value = password.Password(value) - elif isinstance(proptype, hyperdb.Date): - if value: - value = date.Date(form[key].value.strip()) - else: - continue - elif isinstance(proptype, hyperdb.Interval): - if value: - value = date.Interval(form[key].value.strip()) - else: - continue - elif isinstance(proptype, hyperdb.Link): - # see if it's the "no selection" choice - if value == '-1': - value = None - else: - # handle key values - link = proptype.classname - if not num_re.match(value): - try: - value = db.classes[link].lookup(value) - except KeyError: - raise ValueError, _('property "%(propname)s": ' - '%(value)s not a %(classname)s')%{'propname':key, - 'value': value, 'classname': link} - except TypeError, message: - raise ValueError, _('you may only enter ID values ' - 'for property "%(propname)s": %(message)s')%{ - 'propname':key, 'message': message} - elif isinstance(proptype, hyperdb.Multilink): - if isinstance(value, type([])): - # it's a list of MiniFieldStorages - value = [i.value.strip() for i in value] - else: - # it's a MiniFieldStorage, but may be a comma-separated list - # of values - value = [i.strip() for i in value.value.split(',')] - link = proptype.classname - l = [] - for entry in map(str, value): - if entry == '': continue - if not num_re.match(entry): - try: - entry = db.classes[link].lookup(entry) - except KeyError: - raise ValueError, _('property "%(propname)s": ' - '"%(value)s" not an entry of %(classname)s')%{ - 'propname':key, 'value': entry, 'classname': link} - except TypeError, message: - raise ValueError, _('you may only enter ID values ' - 'for property "%(propname)s": %(message)s')%{ - 'propname':key, 'message': message} - l.append(entry) - l.sort() - value = l - elif isinstance(proptype, hyperdb.Boolean): - props[key] = value = value.lower() in ('yes', 'true', 'on', '1') - elif isinstance(proptype, hyperdb.Number): - props[key] = value = int(value) - - # register this as received if required? - if key in required and value is not None: - required.remove(key) - - # get the old value - if nodeid: - try: - existing = cl.get(nodeid, key) - except KeyError: - # this might be a new property for which there is no existing - # value - if not properties.has_key(key): raise - - # if changed, set it - if value != existing: - props[key] = value - else: - props[key] = value - - # see if all the required properties have been supplied - if required: - if len(required) > 1: - p = 'properties' - else: - p = 'property' - raise ValueError, 'Required %s %s not supplied'%(p, ', '.join(required)) - - return props - -
--- a/roundup/cgi/templating.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1542 +0,0 @@ -import sys, cgi, urllib, os, re, os.path, time, errno - -from roundup import hyperdb, date -from roundup.i18n import _ - -try: - import cPickle as pickle -except ImportError: - import pickle -try: - import cStringIO as StringIO -except ImportError: - import StringIO -try: - import StructuredText -except ImportError: - StructuredText = None - -# bring in the templating support -from roundup.cgi.PageTemplates import PageTemplate -from roundup.cgi.PageTemplates.Expressions import getEngine -from roundup.cgi.TAL.TALInterpreter import TALInterpreter -from roundup.cgi import ZTUtils - -class NoTemplate(Exception): - pass - -class Templates: - templates = {} - - def __init__(self, dir): - self.dir = dir - - def precompileTemplates(self): - ''' Go through a directory and precompile all the templates therein - ''' - for filename in os.listdir(self.dir): - if os.path.isdir(filename): continue - if '.' in filename: - name, extension = filename.split('.') - self.getTemplate(name, extension) - else: - self.getTemplate(filename, None) - - def get(self, name, extension): - ''' Interface to get a template, possibly loading a compiled template. - - "name" and "extension" indicate the template we're after, which in - most cases will be "name.extension". If "extension" is None, then - we look for a template just called "name" with no extension. - - If the file "name.extension" doesn't exist, we look for - "_generic.extension" as a fallback. - ''' - # default the name to "home" - if name is None: - name = 'home' - - # find the source, figure the time it was last modified - if extension: - filename = '%s.%s'%(name, extension) - else: - filename = name - src = os.path.join(self.dir, filename) - try: - stime = os.stat(src)[os.path.stat.ST_MTIME] - except os.error, error: - if error.errno != errno.ENOENT: - raise - if not extension: - raise NoTemplate, 'Template file "%s" doesn\'t exist'%name - - # try for a generic template - generic = '_generic.%s'%extension - src = os.path.join(self.dir, generic) - try: - stime = os.stat(src)[os.path.stat.ST_MTIME] - except os.error, error: - if error.errno != errno.ENOENT: - raise - # nicer error - raise NoTemplate, 'No template file exists for templating '\ - '"%s" with template "%s" (neither "%s" nor "%s")'%(name, - extension, filename, generic) - filename = generic - - if self.templates.has_key(filename) and \ - stime < self.templates[filename].mtime: - # compiled template is up to date - return self.templates[filename] - - # compile the template - self.templates[filename] = pt = RoundupPageTemplate() - pt.write(open(src).read()) - pt.id = filename - pt.mtime = time.time() - return pt - - def __getitem__(self, name): - name, extension = os.path.splitext(name) - if extension: - extension = extension[1:] - try: - return self.get(name, extension) - except NoTemplate, message: - raise KeyError, message - -class RoundupPageTemplate(PageTemplate.PageTemplate): - ''' A Roundup-specific PageTemplate. - - Interrogate the client to set up the various template variables to - be available: - - *context* - this is one of three things: - 1. None - we're viewing a "home" page - 2. The current class of item being displayed. This is an HTMLClass - instance. - 3. The current item from the database, if we're viewing a specific - item, as an HTMLItem instance. - *request* - Includes information about the current request, including: - - the url - - the current index information (``filterspec``, ``filter`` args, - ``properties``, etc) parsed out of the form. - - methods for easy filterspec link generation - - *user*, the current user node as an HTMLItem instance - - *form*, the current CGI form information as a FieldStorage - *tracker* - The current tracker - *db* - The current database, through which db.config may be reached. - ''' - def getContext(self, client, classname, request): - c = { - 'options': {}, - 'nothing': None, - 'request': request, - 'db': HTMLDatabase(client), - 'tracker': client.instance, - 'utils': TemplatingUtils(client), - 'templates': Templates(client.instance.config.TEMPLATES), - } - # add in the item if there is one - if client.nodeid: - if classname == 'user': - c['context'] = HTMLUser(client, classname, client.nodeid) - else: - c['context'] = HTMLItem(client, classname, client.nodeid) - elif client.db.classes.has_key(classname): - c['context'] = HTMLClass(client, classname) - return c - - def render(self, client, classname, request, **options): - """Render this Page Template""" - - if not self._v_cooked: - self._cook() - - __traceback_supplement__ = (PageTemplate.PageTemplateTracebackSupplement, self) - - if self._v_errors: - raise PageTemplate.PTRuntimeError, \ - 'Page Template %s has errors.'%self.id - - # figure the context - classname = classname or client.classname - request = request or HTMLRequest(client) - c = self.getContext(client, classname, request) - c.update({'options': options}) - - # and go - output = StringIO.StringIO() - TALInterpreter(self._v_program, self.macros, - getEngine().getContext(c), output, tal=1, strictinsert=0)() - return output.getvalue() - -class HTMLDatabase: - ''' Return HTMLClasses for valid class fetches - ''' - def __init__(self, client): - self._client = client - - # we want config to be exposed - self.config = client.db.config - - def __getitem__(self, item): - self._client.db.getclass(item) - return HTMLClass(self._client, item) - - def __getattr__(self, attr): - try: - return self[attr] - except KeyError: - raise AttributeError, attr - - def classes(self): - l = self._client.db.classes.keys() - l.sort() - return [HTMLClass(self._client, cn) for cn in l] - -def lookupIds(db, prop, ids, num_re=re.compile('-?\d+')): - cl = db.getclass(prop.classname) - l = [] - for entry in ids: - if num_re.match(entry): - l.append(entry) - else: - try: - l.append(cl.lookup(entry)) - except KeyError: - # ignore invalid keys - pass - return l - -class HTMLPermissions: - ''' Helpers that provide answers to commonly asked Permission questions. - ''' - def is_edit_ok(self): - ''' Is the user allowed to Edit the current class? - ''' - return self._db.security.hasPermission('Edit', self._client.userid, - self._classname) - def is_view_ok(self): - ''' Is the user allowed to View the current class? - ''' - return self._db.security.hasPermission('View', self._client.userid, - self._classname) - def is_only_view_ok(self): - ''' Is the user only allowed to View (ie. not Edit) the current class? - ''' - return self.is_view_ok() and not self.is_edit_ok() - -class HTMLClass(HTMLPermissions): - ''' Accesses through a class (either through *class* or *db.<classname>*) - ''' - def __init__(self, client, classname): - self._client = client - self._db = client.db - - # we want classname to be exposed, but _classname gives a - # consistent API for extending Class/Item - self._classname = self.classname = classname - self._klass = self._db.getclass(self.classname) - self._props = self._klass.getprops() - - def __repr__(self): - return '<HTMLClass(0x%x) %s>'%(id(self), self.classname) - - def __getitem__(self, item): - ''' return an HTMLProperty instance - ''' - #print 'HTMLClass.getitem', (self, item) - - # we don't exist - if item == 'id': - return None - - # get the property - prop = self._props[item] - - # look up the correct HTMLProperty class - form = self._client.form - for klass, htmlklass in propclasses: - if not isinstance(prop, klass): - continue - if form.has_key(item): - if isinstance(prop, hyperdb.Multilink): - value = lookupIds(self._db, prop, - handleListCGIValue(form[item])) - elif isinstance(prop, hyperdb.Link): - value = form[item].value.strip() - if value: - value = lookupIds(self._db, prop, [value])[0] - else: - value = None - else: - value = form[item].value.strip() or None - else: - if isinstance(prop, hyperdb.Multilink): - value = [] - else: - value = None - return htmlklass(self._client, '', prop, item, value) - - # no good - raise KeyError, item - - def __getattr__(self, attr): - ''' convenience access ''' - try: - return self[attr] - except KeyError: - raise AttributeError, attr - - def getItem(self, itemid, num_re=re.compile('\d+')): - ''' Get an item of this class by its item id. - ''' - # make sure we're looking at an itemid - if not num_re.match(itemid): - itemid = self._klass.lookup(itemid) - - if self.classname == 'user': - klass = HTMLUser - else: - klass = HTMLItem - - return klass(self._client, self.classname, itemid) - - def properties(self): - ''' Return HTMLProperty for all of this class' properties. - ''' - l = [] - for name, prop in self._props.items(): - for klass, htmlklass in propclasses: - if isinstance(prop, hyperdb.Multilink): - value = [] - else: - value = None - if isinstance(prop, klass): - l.append(htmlklass(self._client, '', prop, name, value)) - return l - - def list(self): - ''' List all items in this class. - ''' - if self.classname == 'user': - klass = HTMLUser - else: - klass = HTMLItem - - # get the list and sort it nicely - l = self._klass.list() - sortfunc = make_sort_function(self._db, self.classname) - l.sort(sortfunc) - - l = [klass(self._client, self.classname, x) for x in l] - return l - - def csv(self): - ''' Return the items of this class as a chunk of CSV text. - ''' - # get the CSV module - try: - import csv - except ImportError: - return 'Sorry, you need the csv module to use this function.\n'\ - 'Get it from: http://www.object-craft.com.au/projects/csv/' - - props = self.propnames() - p = csv.parser() - s = StringIO.StringIO() - s.write(p.join(props) + '\n') - for nodeid in self._klass.list(): - l = [] - for name in props: - value = self._klass.get(nodeid, name) - if value is None: - l.append('') - elif isinstance(value, type([])): - l.append(':'.join(map(str, value))) - else: - l.append(str(self._klass.get(nodeid, name))) - s.write(p.join(l) + '\n') - return s.getvalue() - - def propnames(self): - ''' Return the list of the names of the properties of this class. - ''' - idlessprops = self._klass.getprops(protected=0).keys() - idlessprops.sort() - return ['id'] + idlessprops - - def filter(self, request=None): - ''' Return a list of items from this class, filtered and sorted - by the current requested filterspec/filter/sort/group args - ''' - if request is not None: - filterspec = request.filterspec - sort = request.sort - group = request.group - if self.classname == 'user': - klass = HTMLUser - else: - klass = HTMLItem - l = [klass(self._client, self.classname, x) - for x in self._klass.filter(None, filterspec, sort, group)] - return l - - def classhelp(self, properties=None, label='list', width='500', - height='400'): - ''' Pop up a javascript window with class help - - This generates a link to a popup window which displays the - properties indicated by "properties" of the class named by - "classname". The "properties" should be a comma-separated list - (eg. 'id,name,description'). Properties defaults to all the - properties of a class (excluding id, creator, created and - activity). - - You may optionally override the label displayed, the width and - height. The popup window will be resizable and scrollable. - ''' - if properties is None: - properties = self._klass.getprops(protected=0).keys() - properties.sort() - properties = ','.join(properties) - return '<a href="javascript:help_window(\'%s?:template=help&' \ - 'properties=%s\', \'%s\', \'%s\')"><b>(%s)</b></a>'%( - self.classname, properties, width, height, label) - - def submit(self, label="Submit New Entry"): - ''' Generate a submit button (and action hidden element) - ''' - return ' <input type="hidden" name=":action" value="new">\n'\ - ' <input type="submit" name="submit" value="%s">'%label - - def history(self): - return 'New node - no history' - - def renderWith(self, name, **kwargs): - ''' Render this class with the given template. - ''' - # create a new request and override the specified args - req = HTMLRequest(self._client) - req.classname = self.classname - req.update(kwargs) - - # new template, using the specified classname and request - pt = Templates(self._db.config.TEMPLATES).get(self.classname, name) - - # use our fabricated request - return pt.render(self._client, self.classname, req) - -class HTMLItem(HTMLPermissions): - ''' Accesses through an *item* - ''' - def __init__(self, client, classname, nodeid): - self._client = client - self._db = client.db - self._classname = classname - self._nodeid = nodeid - self._klass = self._db.getclass(classname) - self._props = self._klass.getprops() - - def __repr__(self): - return '<HTMLItem(0x%x) %s %s>'%(id(self), self._classname, - self._nodeid) - - def __getitem__(self, item): - ''' return an HTMLProperty instance - ''' - #print 'HTMLItem.getitem', (self, item) - if item == 'id': - return self._nodeid - - # get the property - prop = self._props[item] - - # get the value, handling missing values - value = self._klass.get(self._nodeid, item, None) - if value is None: - if isinstance(self._props[item], hyperdb.Multilink): - value = [] - - # look up the correct HTMLProperty class - for klass, htmlklass in propclasses: - if isinstance(prop, klass): - return htmlklass(self._client, self._nodeid, prop, item, value) - - raise KeyErorr, item - - def __getattr__(self, attr): - ''' convenience access to properties ''' - try: - return self[attr] - except KeyError: - raise AttributeError, attr - - def submit(self, label="Submit Changes"): - ''' Generate a submit button (and action hidden element) - ''' - return ' <input type="hidden" name=":action" value="edit">\n'\ - ' <input type="submit" name="submit" value="%s">'%label - - def journal(self, direction='descending'): - ''' Return a list of HTMLJournalEntry instances. - ''' - # XXX do this - return [] - - def history(self, direction='descending'): - l = ['<table class="history">' - '<tr><th colspan="4" class="header">', - _('History'), - '</th></tr><tr>', - _('<th>Date</th>'), - _('<th>User</th>'), - _('<th>Action</th>'), - _('<th>Args</th>'), - '</tr>'] - comments = {} - history = self._klass.history(self._nodeid) - history.sort() - if direction == 'descending': - history.reverse() - for id, evt_date, user, action, args in history: - date_s = str(evt_date).replace("."," ") - arg_s = '' - if action == 'link' and type(args) == type(()): - if len(args) == 3: - linkcl, linkid, key = args - arg_s += '<a href="%s%s">%s%s %s</a>'%(linkcl, linkid, - linkcl, linkid, key) - else: - arg_s = str(args) - - elif action == 'unlink' and type(args) == type(()): - if len(args) == 3: - linkcl, linkid, key = args - arg_s += '<a href="%s%s">%s%s %s</a>'%(linkcl, linkid, - linkcl, linkid, key) - else: - arg_s = str(args) - - elif type(args) == type({}): - cell = [] - for k in args.keys(): - # try to get the relevant property and treat it - # specially - try: - prop = self._props[k] - except KeyError: - prop = None - if prop is not None: - if args[k] and (isinstance(prop, hyperdb.Multilink) or - isinstance(prop, hyperdb.Link)): - # figure what the link class is - classname = prop.classname - try: - linkcl = self._db.getclass(classname) - except KeyError: - labelprop = None - comments[classname] = _('''The linked class - %(classname)s no longer exists''')%locals() - labelprop = linkcl.labelprop(1) - hrefable = os.path.exists( - os.path.join(self._db.config.TEMPLATES, - classname+'.item')) - - if isinstance(prop, hyperdb.Multilink) and \ - len(args[k]) > 0: - ml = [] - for linkid in args[k]: - if isinstance(linkid, type(())): - sublabel = linkid[0] + ' ' - linkids = linkid[1] - else: - sublabel = '' - linkids = [linkid] - subml = [] - for linkid in linkids: - label = classname + linkid - # if we have a label property, try to use it - # TODO: test for node existence even when - # there's no labelprop! - try: - if labelprop is not None: - label = linkcl.get(linkid, labelprop) - except IndexError: - comments['no_link'] = _('''<strike>The - linked node no longer - exists</strike>''') - subml.append('<strike>%s</strike>'%label) - else: - if hrefable: - subml.append('<a href="%s%s">%s</a>'%( - classname, linkid, label)) - ml.append(sublabel + ', '.join(subml)) - cell.append('%s:\n %s'%(k, ', '.join(ml))) - elif isinstance(prop, hyperdb.Link) and args[k]: - label = classname + args[k] - # if we have a label property, try to use it - # TODO: test for node existence even when - # there's no labelprop! - if labelprop is not None: - try: - label = linkcl.get(args[k], labelprop) - except IndexError: - comments['no_link'] = _('''<strike>The - linked node no longer - exists</strike>''') - cell.append(' <strike>%s</strike>,\n'%label) - # "flag" this is done .... euwww - label = None - if label is not None: - if hrefable: - cell.append('%s: <a href="%s%s">%s</a>\n'%(k, - classname, args[k], label)) - else: - cell.append('%s: %s' % (k,label)) - - elif isinstance(prop, hyperdb.Date) and args[k]: - d = date.Date(args[k]) - cell.append('%s: %s'%(k, str(d))) - - elif isinstance(prop, hyperdb.Interval) and args[k]: - d = date.Interval(args[k]) - cell.append('%s: %s'%(k, str(d))) - - elif isinstance(prop, hyperdb.String) and args[k]: - cell.append('%s: %s'%(k, cgi.escape(args[k]))) - - elif not args[k]: - cell.append('%s: (no value)\n'%k) - - else: - cell.append('%s: %s\n'%(k, str(args[k]))) - else: - # property no longer exists - comments['no_exist'] = _('''<em>The indicated property - no longer exists</em>''') - cell.append('<em>%s: %s</em>\n'%(k, str(args[k]))) - arg_s = '<br />'.join(cell) - else: - # unkown event!! - comments['unknown'] = _('''<strong><em>This event is not - handled by the history display!</em></strong>''') - arg_s = '<strong><em>' + str(args) + '</em></strong>' - date_s = date_s.replace(' ', ' ') - l.append('<tr><td>%s</td><td>%s</td><td>%s</td><td>%s</td></tr>'%( - date_s, user, action, arg_s)) - if comments: - l.append(_('<tr><td colspan=4><strong>Note:</strong></td></tr>')) - for entry in comments.values(): - l.append('<tr><td colspan=4>%s</td></tr>'%entry) - l.append('</table>') - return '\n'.join(l) - - def renderQueryForm(self): - ''' Render this item, which is a query, as a search form. - ''' - # create a new request and override the specified args - req = HTMLRequest(self._client) - req.classname = self._klass.get(self._nodeid, 'klass') - req.updateFromURL(self._klass.get(self._nodeid, 'url')) - - # new template, using the specified classname and request - pt = getTemplate(self._db.config.TEMPLATES, req.classname, 'search') - - # use our fabricated request - return pt.render(self._client, req.classname, req) - -class HTMLUser(HTMLItem): - ''' Accesses through the *user* (a special case of item) - ''' - def __init__(self, client, classname, nodeid): - HTMLItem.__init__(self, client, 'user', nodeid) - self._default_classname = client.classname - - # used for security checks - self._security = client.db.security - - _marker = [] - def hasPermission(self, role, classname=_marker): - ''' Determine if the user has the Role. - - The class being tested defaults to the template's class, but may - be overidden for this test by suppling an alternate classname. - ''' - if classname is self._marker: - classname = self._default_classname - return self._security.hasPermission(role, self._nodeid, classname) - - def is_edit_ok(self): - ''' Is the user allowed to Edit the current class? - Also check whether this is the current user's info. - ''' - return self._db.security.hasPermission('Edit', self._client.userid, - self._classname) or self._nodeid == self._client.userid - - def is_view_ok(self): - ''' Is the user allowed to View the current class? - Also check whether this is the current user's info. - ''' - return self._db.security.hasPermission('Edit', self._client.userid, - self._classname) or self._nodeid == self._client.userid - -class HTMLProperty: - ''' String, Number, Date, Interval HTMLProperty - - Has useful attributes: - - _name the name of the property - _value the value of the property if any - - A wrapper object which may be stringified for the plain() behaviour. - ''' - def __init__(self, client, nodeid, prop, name, value): - self._client = client - self._db = client.db - self._nodeid = nodeid - self._prop = prop - self._name = name - self._value = value - def __repr__(self): - return '<HTMLProperty(0x%x) %s %r %r>'%(id(self), self._name, self._prop, self._value) - def __str__(self): - return self.plain() - def __cmp__(self, other): - if isinstance(other, HTMLProperty): - return cmp(self._value, other._value) - return cmp(self._value, other) - -class StringHTMLProperty(HTMLProperty): - def plain(self, escape=0): - ''' Render a "plain" representation of the property - ''' - if self._value is None: - return '' - if escape: - return cgi.escape(str(self._value)) - return str(self._value) - - def stext(self, escape=0): - ''' Render the value of the property as StructuredText. - - This requires the StructureText module to be installed separately. - ''' - s = self.plain(escape=escape) - if not StructuredText: - return s - return StructuredText(s,level=1,header=0) - - def field(self, size = 30): - ''' Render a form edit field for the property - ''' - if self._value is None: - value = '' - else: - value = cgi.escape(str(self._value)) - value = '"'.join(value.split('"')) - return '<input name="%s" value="%s" size="%s">'%(self._name, value, size) - - def multiline(self, escape=0, rows=5, cols=40): - ''' Render a multiline form edit field for the property - ''' - if self._value is None: - value = '' - else: - value = cgi.escape(str(self._value)) - value = '"'.join(value.split('"')) - return '<textarea name="%s" rows="%s" cols="%s">%s</textarea>'%( - self._name, rows, cols, value) - - def email(self, escape=1): - ''' Render the value of the property as an obscured email address - ''' - if self._value is None: value = '' - else: value = str(self._value) - if value.find('@') != -1: - name, domain = value.split('@') - domain = ' '.join(domain.split('.')[:-1]) - name = name.replace('.', ' ') - value = '%s at %s ...'%(name, domain) - else: - value = value.replace('.', ' ') - if escape: - value = cgi.escape(value) - return value - -class PasswordHTMLProperty(HTMLProperty): - def plain(self): - ''' Render a "plain" representation of the property - ''' - if self._value is None: - return '' - return _('*encrypted*') - - def field(self, size = 30): - ''' Render a form edit field for the property. - ''' - return '<input type="password" name="%s" size="%s">'%(self._name, size) - - def confirm(self, size = 30): - ''' Render a second form edit field for the property, used for - confirmation that the user typed the password correctly. Generates - a field with name "name:confirm". - ''' - return '<input type="password" name="%s:confirm" size="%s">'%( - self._name, size) - -class NumberHTMLProperty(HTMLProperty): - def plain(self): - ''' Render a "plain" representation of the property - ''' - return str(self._value) - - def field(self, size = 30): - ''' Render a form edit field for the property - ''' - if self._value is None: - value = '' - else: - value = cgi.escape(str(self._value)) - value = '"'.join(value.split('"')) - return '<input name="%s" value="%s" size="%s">'%(self._name, value, size) - -class BooleanHTMLProperty(HTMLProperty): - def plain(self): - ''' Render a "plain" representation of the property - ''' - if self.value is None: - return '' - return self._value and "Yes" or "No" - - def field(self): - ''' Render a form edit field for the property - ''' - checked = self._value and "checked" or "" - s = '<input type="radio" name="%s" value="yes" %s>Yes'%(self._name, - checked) - if checked: - checked = "" - else: - checked = "checked" - s += '<input type="radio" name="%s" value="no" %s>No'%(self._name, - checked) - return s - -class DateHTMLProperty(HTMLProperty): - def plain(self): - ''' Render a "plain" representation of the property - ''' - if self._value is None: - return '' - return str(self._value) - - def field(self, size = 30): - ''' Render a form edit field for the property - ''' - if self._value is None: - value = '' - else: - value = cgi.escape(str(self._value)) - value = '"'.join(value.split('"')) - return '<input name="%s" value="%s" size="%s">'%(self._name, value, size) - - def reldate(self, pretty=1): - ''' Render the interval between the date and now. - - If the "pretty" flag is true, then make the display pretty. - ''' - if not self._value: - return '' - - # figure the interval - interval = date.Date('.') - self._value - if pretty: - return interval.pretty() - return str(interval) - -class IntervalHTMLProperty(HTMLProperty): - def plain(self): - ''' Render a "plain" representation of the property - ''' - if self._value is None: - return '' - return str(self._value) - - def pretty(self): - ''' Render the interval in a pretty format (eg. "yesterday") - ''' - return self._value.pretty() - - def field(self, size = 30): - ''' Render a form edit field for the property - ''' - if self._value is None: - value = '' - else: - value = cgi.escape(str(self._value)) - value = '"'.join(value.split('"')) - return '<input name="%s" value="%s" size="%s">'%(self._name, value, size) - -class LinkHTMLProperty(HTMLProperty): - ''' Link HTMLProperty - Include the above as well as being able to access the class - information. Stringifying the object itself results in the value - from the item being displayed. Accessing attributes of this object - result in the appropriate entry from the class being queried for the - property accessed (so item/assignedto/name would look up the user - entry identified by the assignedto property on item, and then the - name property of that user) - ''' - def __getattr__(self, attr): - ''' return a new HTMLItem ''' - #print 'Link.getattr', (self, attr, self._value) - if not self._value: - raise AttributeError, "Can't access missing value" - if self._prop.classname == 'user': - klass = HTMLUser - else: - klass = HTMLItem - i = klass(self._client, self._prop.classname, self._value) - return getattr(i, attr) - - def plain(self, escape=0): - ''' Render a "plain" representation of the property - ''' - if self._value is None: - return '' - linkcl = self._db.classes[self._prop.classname] - k = linkcl.labelprop(1) - value = str(linkcl.get(self._value, k)) - if escape: - value = cgi.escape(value) - return value - - def field(self, showid=0, size=None): - ''' Render a form edit field for the property - ''' - linkcl = self._db.getclass(self._prop.classname) - if linkcl.getprops().has_key('order'): - sort_on = 'order' - else: - sort_on = linkcl.labelprop() - options = linkcl.filter(None, {}, ('+', sort_on), (None, None)) - # TODO: make this a field display, not a menu one! - l = ['<select name="%s">'%self._name] - k = linkcl.labelprop(1) - if self._value is None: - s = 'selected ' - else: - s = '' - l.append(_('<option %svalue="-1">- no selection -</option>')%s) - for optionid in options: - # get the option value, and if it's None use an empty string - option = linkcl.get(optionid, k) or '' - - # figure if this option is selected - s = '' - if optionid == self._value: - s = 'selected ' - - # figure the label - if showid: - lab = '%s%s: %s'%(self._prop.classname, optionid, option) - else: - lab = option - - # truncate if it's too long - if size is not None and len(lab) > size: - lab = lab[:size-3] + '...' - - # and generate - lab = cgi.escape(lab) - l.append('<option %svalue="%s">%s</option>'%(s, optionid, lab)) - l.append('</select>') - return '\n'.join(l) - - def menu(self, size=None, height=None, showid=0, additional=[], - **conditions): - ''' Render a form select list for this property - ''' - value = self._value - - # sort function - sortfunc = make_sort_function(self._db, self._prop.classname) - - linkcl = self._db.getclass(self._prop.classname) - l = ['<select name="%s">'%self._name] - k = linkcl.labelprop(1) - s = '' - if value is None: - s = 'selected ' - l.append(_('<option %svalue="-1">- no selection -</option>')%s) - if linkcl.getprops().has_key('order'): - sort_on = ('+', 'order') - else: - sort_on = ('+', linkcl.labelprop()) - options = linkcl.filter(None, conditions, sort_on, (None, None)) - for optionid in options: - # get the option value, and if it's None use an empty string - option = linkcl.get(optionid, k) or '' - - # figure if this option is selected - s = '' - if value in [optionid, option]: - s = 'selected ' - - # figure the label - if showid: - lab = '%s%s: %s'%(self._prop.classname, optionid, option) - else: - lab = option - - # truncate if it's too long - if size is not None and len(lab) > size: - lab = lab[:size-3] + '...' - if additional: - m = [] - for propname in additional: - m.append(linkcl.get(optionid, propname)) - lab = lab + ' (%s)'%', '.join(map(str, m)) - - # and generate - lab = cgi.escape(lab) - l.append('<option %svalue="%s">%s</option>'%(s, optionid, lab)) - l.append('</select>') - return '\n'.join(l) -# def checklist(self, ...) - -class MultilinkHTMLProperty(HTMLProperty): - ''' Multilink HTMLProperty - - Also be iterable, returning a wrapper object like the Link case for - each entry in the multilink. - ''' - def __len__(self): - ''' length of the multilink ''' - return len(self._value) - - def __getattr__(self, attr): - ''' no extended attribute accesses make sense here ''' - raise AttributeError, attr - - def __getitem__(self, num): - ''' iterate and return a new HTMLItem - ''' - #print 'Multi.getitem', (self, num) - value = self._value[num] - if self._prop.classname == 'user': - klass = HTMLUser - else: - klass = HTMLItem - return klass(self._client, self._prop.classname, value) - - def __contains__(self, value): - ''' Support the "in" operator - ''' - return value in self._value - - def reverse(self): - ''' return the list in reverse order - ''' - l = self._value[:] - l.reverse() - if self._prop.classname == 'user': - klass = HTMLUser - else: - klass = HTMLItem - return [klass(self._client, self._prop.classname, value) for value in l] - - def plain(self, escape=0): - ''' Render a "plain" representation of the property - ''' - linkcl = self._db.classes[self._prop.classname] - k = linkcl.labelprop(1) - labels = [] - for v in self._value: - labels.append(linkcl.get(v, k)) - value = ', '.join(labels) - if escape: - value = cgi.escape(value) - return value - - def field(self, size=30, showid=0): - ''' Render a form edit field for the property - ''' - sortfunc = make_sort_function(self._db, self._prop.classname) - linkcl = self._db.getclass(self._prop.classname) - value = self._value[:] - if value: - value.sort(sortfunc) - # map the id to the label property - if not linkcl.getkey(): - showid=1 - if not showid: - k = linkcl.labelprop(1) - value = [linkcl.get(v, k) for v in value] - value = cgi.escape(','.join(value)) - return '<input name="%s" size="%s" value="%s">'%(self._name, size, value) - - def menu(self, size=None, height=None, showid=0, additional=[], - **conditions): - ''' Render a form select list for this property - ''' - value = self._value - - # sort function - sortfunc = make_sort_function(self._db, self._prop.classname) - - linkcl = self._db.getclass(self._prop.classname) - if linkcl.getprops().has_key('order'): - sort_on = ('+', 'order') - else: - sort_on = ('+', linkcl.labelprop()) - options = linkcl.filter(None, conditions, sort_on, (None,None)) - height = height or min(len(options), 7) - l = ['<select multiple name="%s" size="%s">'%(self._name, height)] - k = linkcl.labelprop(1) - for optionid in options: - # get the option value, and if it's None use an empty string - option = linkcl.get(optionid, k) or '' - - # figure if this option is selected - s = '' - if optionid in value or option in value: - s = 'selected ' - - # figure the label - if showid: - lab = '%s%s: %s'%(self._prop.classname, optionid, option) - else: - lab = option - # truncate if it's too long - if size is not None and len(lab) > size: - lab = lab[:size-3] + '...' - if additional: - m = [] - for propname in additional: - m.append(linkcl.get(optionid, propname)) - lab = lab + ' (%s)'%', '.join(m) - - # and generate - lab = cgi.escape(lab) - l.append('<option %svalue="%s">%s</option>'%(s, optionid, - lab)) - l.append('</select>') - return '\n'.join(l) - -# set the propclasses for HTMLItem -propclasses = ( - (hyperdb.String, StringHTMLProperty), - (hyperdb.Number, NumberHTMLProperty), - (hyperdb.Boolean, BooleanHTMLProperty), - (hyperdb.Date, DateHTMLProperty), - (hyperdb.Interval, IntervalHTMLProperty), - (hyperdb.Password, PasswordHTMLProperty), - (hyperdb.Link, LinkHTMLProperty), - (hyperdb.Multilink, MultilinkHTMLProperty), -) - -def make_sort_function(db, classname): - '''Make a sort function for a given class - ''' - linkcl = db.getclass(classname) - if linkcl.getprops().has_key('order'): - sort_on = 'order' - else: - sort_on = linkcl.labelprop() - def sortfunc(a, b, linkcl=linkcl, sort_on=sort_on): - return cmp(linkcl.get(a, sort_on), linkcl.get(b, sort_on)) - return sortfunc - -def handleListCGIValue(value): - ''' Value is either a single item or a list of items. Each item has a - .value that we're actually interested in. - ''' - if isinstance(value, type([])): - return [value.value for value in value] - else: - value = value.value.strip() - if not value: - return [] - return value.split(',') - -class ShowDict: - ''' A convenience access to the :columns index parameters - ''' - def __init__(self, columns): - self.columns = {} - for col in columns: - self.columns[col] = 1 - def __getitem__(self, name): - return self.columns.has_key(name) - -class HTMLRequest: - ''' The *request*, holding the CGI form and environment. - - "form" the CGI form as a cgi.FieldStorage - "env" the CGI environment variables - "base" the base URL for this instance - "user" a HTMLUser instance for this user - "classname" the current classname (possibly None) - "template" the current template (suffix, also possibly None) - - Index args: - "columns" dictionary of the columns to display in an index page - "show" a convenience access to columns - request/show/colname will - be true if the columns should be displayed, false otherwise - "sort" index sort column (direction, column name) - "group" index grouping property (direction, column name) - "filter" properties to filter the index on - "filterspec" values to filter the index on - "search_text" text to perform a full-text search on for an index - - ''' - def __init__(self, client): - self.client = client - - # easier access vars - self.form = client.form - self.env = client.env - self.base = client.base - self.user = HTMLUser(client, 'user', client.userid) - - # store the current class name and action - self.classname = client.classname - self.template = client.template - - self._post_init() - - def _post_init(self): - ''' Set attributes based on self.form - ''' - # extract the index display information from the form - self.columns = [] - if self.form.has_key(':columns'): - self.columns = handleListCGIValue(self.form[':columns']) - self.show = ShowDict(self.columns) - - # sorting - self.sort = (None, None) - if self.form.has_key(':sort'): - sort = self.form[':sort'].value - if sort.startswith('-'): - self.sort = ('-', sort[1:]) - else: - self.sort = ('+', sort) - if self.form.has_key(':sortdir'): - self.sort = ('-', self.sort[1]) - - # grouping - self.group = (None, None) - if self.form.has_key(':group'): - group = self.form[':group'].value - if group.startswith('-'): - self.group = ('-', group[1:]) - else: - self.group = ('+', group) - if self.form.has_key(':groupdir'): - self.group = ('-', self.group[1]) - - # filtering - self.filter = [] - if self.form.has_key(':filter'): - self.filter = handleListCGIValue(self.form[':filter']) - self.filterspec = {} - db = self.client.db - if self.classname is not None: - props = db.getclass(self.classname).getprops() - for name in self.filter: - if self.form.has_key(name): - prop = props[name] - fv = self.form[name] - if (isinstance(prop, hyperdb.Link) or - isinstance(prop, hyperdb.Multilink)): - self.filterspec[name] = lookupIds(db, prop, - handleListCGIValue(fv)) - else: - self.filterspec[name] = fv.value - - # full-text search argument - self.search_text = None - if self.form.has_key(':search_text'): - self.search_text = self.form[':search_text'].value - - # pagination - size and start index - # figure batch args - if self.form.has_key(':pagesize'): - self.pagesize = int(self.form[':pagesize'].value) - else: - self.pagesize = 50 - if self.form.has_key(':startwith'): - self.startwith = int(self.form[':startwith'].value) - else: - self.startwith = 0 - - def updateFromURL(self, url): - ''' Parse the URL for query args, and update my attributes using the - values. - ''' - self.form = {} - for name, value in cgi.parse_qsl(url): - if self.form.has_key(name): - if isinstance(self.form[name], type([])): - self.form[name].append(cgi.MiniFieldStorage(name, value)) - else: - self.form[name] = [self.form[name], - cgi.MiniFieldStorage(name, value)] - else: - self.form[name] = cgi.MiniFieldStorage(name, value) - self._post_init() - - def update(self, kwargs): - ''' Update my attributes using the keyword args - ''' - self.__dict__.update(kwargs) - if kwargs.has_key('columns'): - self.show = ShowDict(self.columns) - - def description(self): - ''' Return a description of the request - handle for the page title. - ''' - s = [self.client.db.config.TRACKER_NAME] - if self.classname: - if self.client.nodeid: - s.append('- %s%s'%(self.classname, self.client.nodeid)) - else: - if self.template == 'item': - s.append('- new %s'%self.classname) - elif self.template == 'index': - s.append('- %s index'%self.classname) - else: - s.append('- %s %s'%(self.classname, self.template)) - else: - s.append('- home') - return ' '.join(s) - - def __str__(self): - d = {} - d.update(self.__dict__) - f = '' - for k in self.form.keys(): - f += '\n %r=%r'%(k,handleListCGIValue(self.form[k])) - d['form'] = f - e = '' - for k,v in self.env.items(): - e += '\n %r=%r'%(k, v) - d['env'] = e - return ''' -form: %(form)s -base: %(base)r -classname: %(classname)r -template: %(template)r -columns: %(columns)r -sort: %(sort)r -group: %(group)r -filter: %(filter)r -search_text: %(search_text)r -pagesize: %(pagesize)r -startwith: %(startwith)r -env: %(env)s -'''%d - - def indexargs_form(self, columns=1, sort=1, group=1, filter=1, - filterspec=1): - ''' return the current index args as form elements ''' - l = [] - s = '<input type="hidden" name="%s" value="%s">' - if columns and self.columns: - l.append(s%(':columns', ','.join(self.columns))) - if sort and self.sort[1] is not None: - if self.sort[0] == '-': - val = '-'+self.sort[1] - else: - val = self.sort[1] - l.append(s%(':sort', val)) - if group and self.group[1] is not None: - if self.group[0] == '-': - val = '-'+self.group[1] - else: - val = self.group[1] - l.append(s%(':group', val)) - if filter and self.filter: - l.append(s%(':filter', ','.join(self.filter))) - if filterspec: - for k,v in self.filterspec.items(): - l.append(s%(k, ','.join(v))) - if self.search_text: - l.append(s%(':search_text', self.search_text)) - l.append(s%(':pagesize', self.pagesize)) - l.append(s%(':startwith', self.startwith)) - return '\n'.join(l) - - def indexargs_url(self, url, args): - ''' embed the current index args in a URL ''' - l = ['%s=%s'%(k,v) for k,v in args.items()] - if self.columns and not args.has_key(':columns'): - l.append(':columns=%s'%(','.join(self.columns))) - if self.sort[1] is not None and not args.has_key(':sort'): - if self.sort[0] == '-': - val = '-'+self.sort[1] - else: - val = self.sort[1] - l.append(':sort=%s'%val) - if self.group[1] is not None and not args.has_key(':group'): - if self.group[0] == '-': - val = '-'+self.group[1] - else: - val = self.group[1] - l.append(':group=%s'%val) - if self.filter and not args.has_key(':columns'): - l.append(':filter=%s'%(','.join(self.filter))) - for k,v in self.filterspec.items(): - if not args.has_key(k): - l.append('%s=%s'%(k, ','.join(v))) - if self.search_text and not args.has_key(':search_text'): - l.append(':search_text=%s'%self.search_text) - if not args.has_key(':pagesize'): - l.append(':pagesize=%s'%self.pagesize) - if not args.has_key(':startwith'): - l.append(':startwith=%s'%self.startwith) - return '%s?%s'%(url, '&'.join(l)) - indexargs_href = indexargs_url - - def base_javascript(self): - return ''' -<script language="javascript"> -submitted = false; -function submit_once() { - if (submitted) { - alert("Your request is being processed.\\nPlease be patient."); - return 0; - } - submitted = true; - return 1; -} - -function help_window(helpurl, width, height) { - HelpWin = window.open('%s' + helpurl, 'RoundupHelpWindow', 'scrollbars=yes,resizable=yes,toolbar=no,height='+height+',width='+width); -} -</script> -'''%self.base - - def batch(self): - ''' Return a batch object for results from the "current search" - ''' - filterspec = self.filterspec - sort = self.sort - group = self.group - - # get the list of ids we're batching over - klass = self.client.db.getclass(self.classname) - if self.search_text: - matches = self.client.db.indexer.search( - re.findall(r'\b\w{2,25}\b', self.search_text), klass) - else: - matches = None - l = klass.filter(matches, filterspec, sort, group) - - # return the batch object, using IDs only - return Batch(self.client, l, self.pagesize, self.startwith, - classname=self.classname) - -# extend the standard ZTUtils Batch object to remove dependency on -# Acquisition and add a couple of useful methods -class Batch(ZTUtils.Batch): - ''' Use me to turn a list of items, or item ids of a given class, into a - series of batches. - - ========= ======================================================== - Parameter Usage - ========= ======================================================== - sequence a list of HTMLItems or item ids - classname if sequence is a list of ids, this is the class of item - size how big to make the sequence. - start where to start (0-indexed) in the sequence. - end where to end (0-indexed) in the sequence. - orphan if the next batch would contain less items than this - value, then it is combined with this batch - overlap the number of items shared between adjacent batches - ========= ======================================================== - - Attributes: Note that the "start" attribute, unlike the - argument, is a 1-based index (I know, lame). "first" is the - 0-based index. "length" is the actual number of elements in - the batch. - - "sequence_length" is the length of the original, unbatched, sequence. - ''' - def __init__(self, client, sequence, size, start, end=0, orphan=0, - overlap=0, classname=None): - self.client = client - self.last_index = self.last_item = None - self.current_item = None - self.classname = classname - self.sequence_length = len(sequence) - ZTUtils.Batch.__init__(self, sequence, size, start, end, orphan, - overlap) - - # overwrite so we can late-instantiate the HTMLItem instance - def __getitem__(self, index): - if index < 0: - if index + self.end < self.first: raise IndexError, index - return self._sequence[index + self.end] - - if index >= self.length: - raise IndexError, index - - # move the last_item along - but only if the fetched index changes - # (for some reason, index 0 is fetched twice) - if index != self.last_index: - self.last_item = self.current_item - self.last_index = index - - item = self._sequence[index + self.first] - if self.classname: - # map the item ids to instances - if self.classname == 'user': - item = HTMLUser(self.client, self.classname, item) - else: - item = HTMLItem(self.client, self.classname, item) - self.current_item = item - return item - - def propchanged(self, property): - ''' Detect if the property marked as being the group property - changed in the last iteration fetch - ''' - if (self.last_item is None or - self.last_item[property] != self.current_item[property]): - return 1 - return 0 - - # override these 'cos we don't have access to acquisition - def previous(self): - if self.start == 1: - return None - return Batch(self.client, self._sequence, self._size, - self.first - self._size + self.overlap, 0, self.orphan, - self.overlap) - - def next(self): - try: - self._sequence[self.end] - except IndexError: - return None - return Batch(self.client, self._sequence, self._size, - self.end - self.overlap, 0, self.orphan, self.overlap) - -class TemplatingUtils: - ''' Utilities for templating - ''' - def __init__(self, client): - self.client = client - def Batch(self, sequence, size, start, end=0, orphan=0, overlap=0): - return Batch(self.client, sequence, size, start, end, orphan, - overlap) -
--- a/roundup/date.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,473 +0,0 @@ -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: date.py,v 1.32 2002-09-23 12:09:29 richard Exp $ - -__doc__ = """ -Date, time and time interval handling. -""" - -import time, re, calendar -from i18n import _ - -class Date: - ''' - As strings, date-and-time stamps are specified with the date in - international standard format (yyyy-mm-dd) joined to the time - (hh:mm:ss) by a period ("."). Dates in this form can be easily compared - and are fairly readable when printed. An example of a valid stamp is - "2000-06-24.13:03:59". We'll call this the "full date format". When - Timestamp objects are printed as strings, they appear in the full date - format with the time always given in GMT. The full date format is - always exactly 19 characters long. - - For user input, some partial forms are also permitted: the whole time - or just the seconds may be omitted; and the whole date may be omitted - or just the year may be omitted. If the time is given, the time is - interpreted in the user's local time zone. The Date constructor takes - care of these conversions. In the following examples, suppose that yyyy - is the current year, mm is the current month, and dd is the current day - of the month; and suppose that the user is on Eastern Standard Time. - - "2000-04-17" means <Date 2000-04-17.00:00:00> - "01-25" means <Date yyyy-01-25.00:00:00> - "2000-04-17.03:45" means <Date 2000-04-17.08:45:00> - "08-13.22:13" means <Date yyyy-08-14.03:13:00> - "11-07.09:32:43" means <Date yyyy-11-07.14:32:43> - "14:25" means <Date yyyy-mm-dd.19:25:00> - "8:47:11" means <Date yyyy-mm-dd.13:47:11> - "." means "right now" - - The Date class should understand simple date expressions of the form - stamp + interval and stamp - interval. When adding or subtracting - intervals involving months or years, the components are handled - separately. For example, when evaluating "2000-06-25 + 1m 10d", we - first add one month to get 2000-07-25, then add 10 days to get - 2000-08-04 (rather than trying to decide whether 1m 10d means 38 or 40 - or 41 days). - - Example usage: - >>> Date(".") - <Date 2000-06-26.00:34:02> - >>> _.local(-5) - "2000-06-25.19:34:02" - >>> Date(". + 2d") - <Date 2000-06-28.00:34:02> - >>> Date("1997-04-17", -5) - <Date 1997-04-17.00:00:00> - >>> Date("01-25", -5) - <Date 2000-01-25.00:00:00> - >>> Date("08-13.22:13", -5) - <Date 2000-08-14.03:13:00> - >>> Date("14:25", -5) - <Date 2000-06-25.19:25:00> - - The date format 'yyyymmddHHMMSS' (year, month, day, hour, - minute, second) is the serialisation format returned by the serialise() - method, and is accepted as an argument on instatiation. - ''' - def __init__(self, spec='.', offset=0): - """Construct a date given a specification and a time zone offset. - - 'spec' is a full date or a partial form, with an optional - added or subtracted interval. Or a date 9-tuple. - 'offset' is the local time zone offset from GMT in hours. - """ - if type(spec) == type(''): - self.set(spec, offset=offset) - else: - y,m,d,H,M,S,x,x,x = spec - ts = calendar.timegm((y,m,d,H+offset,M,S,0,0,0)) - self.year, self.month, self.day, self.hour, self.minute, \ - self.second, x, x, x = time.gmtime(ts) - - def addInterval(self, interval): - ''' Add the interval to this date, returning the date tuple - ''' - # do the basic calc - sign = interval.sign - year = self.year + sign * interval.year - month = self.month + sign * interval.month - day = self.day + sign * interval.day - hour = self.hour + sign * interval.hour - minute = self.minute + sign * interval.minute - second = self.second + sign * interval.second - - # now cope with under- and over-flow - # first do the time - while (second < 0 or second > 59 or minute < 0 or minute > 59 or - hour < 0 or hour > 59): - if second < 0: minute -= 1; second += 60 - elif second > 59: minute += 1; second -= 60 - if minute < 0: hour -= 1; minute += 60 - elif minute > 59: hour += 1; minute -= 60 - if hour < 0: day -= 1; hour += 24 - elif hour > 59: day += 1; hour -= 24 - - # fix up the month so we're within range - while month < 1 or month > 12: - if month < 1: year -= 1; month += 12 - if month > 12: year += 1; month -= 12 - - # now do the days, now that we know what month we're in - mdays = calendar.mdays - if month == 2 and calendar.isleap(year): month_days = 29 - else: month_days = mdays[month] - while month < 1 or month > 12 or day < 0 or day > month_days: - # now to day under/over - if day < 0: month -= 1; day += month_days - elif day > month_days: month += 1; day -= month_days - - # possibly fix up the month so we're within range - while month < 1 or month > 12: - if month < 1: year -= 1; month += 12 - if month > 12: year += 1; month -= 12 - - # re-figure the number of days for this month - if month == 2 and calendar.isleap(year): month_days = 29 - else: month_days = mdays[month] - return (year, month, day, hour, minute, second, 0, 0, 0) - - def applyInterval(self, interval): - ''' Apply the interval to this date - ''' - self.year, self.month, self.day, self.hour, self.minute, \ - self.second, x, x, x = self.addInterval(interval) - - def __add__(self, interval): - """Add an interval to this date to produce another date. - """ - return Date(self.addInterval(interval)) - - # deviates from spec to allow subtraction of dates as well - def __sub__(self, other): - """ Subtract: - 1. an interval from this date to produce another date. - 2. a date from this date to produce an interval. - """ - if isinstance(other, Interval): - other = Interval(other.get_tuple()) - other.sign *= -1 - return self.__add__(other) - - assert isinstance(other, Date), 'May only subtract Dates or Intervals' - - # TODO this code will fall over laughing if the dates cross - # leap years, phases of the moon, .... - a = calendar.timegm((self.year, self.month, self.day, self.hour, - self.minute, self.second, 0, 0, 0)) - b = calendar.timegm((other.year, other.month, other.day, - other.hour, other.minute, other.second, 0, 0, 0)) - diff = a - b - if diff < 0: - sign = 1 - diff = -diff - else: - sign = -1 - S = diff%60 - M = (diff/60)%60 - H = (diff/(60*60))%60 - if H>1: S = 0 - d = (diff/(24*60*60))%30 - if d>1: H = S = M = 0 - m = (diff/(30*24*60*60))%12 - if m>1: H = S = M = 0 - y = (diff/(365*24*60*60)) - if y>1: d = H = S = M = 0 - return Interval((y, m, d, H, M, S), sign=sign) - - def __cmp__(self, other): - """Compare this date to another date.""" - if other is None: - return 1 - for attr in ('year', 'month', 'day', 'hour', 'minute', 'second'): - if not hasattr(other, attr): - return 1 - r = cmp(getattr(self, attr), getattr(other, attr)) - if r: return r - return 0 - - def __str__(self): - """Return this date as a string in the yyyy-mm-dd.hh:mm:ss format.""" - return '%4d-%02d-%02d.%02d:%02d:%02d'%(self.year, self.month, self.day, - self.hour, self.minute, self.second) - - def pretty(self): - ''' print up the date date using a pretty format... - ''' - str = time.strftime('%d %B %Y', (self.year, self.month, - self.day, self.hour, self.minute, self.second, 0, 0, 0)) - if str[0] == '0': return ' ' + str[1:] - return str - - def set(self, spec, offset=0, date_re=re.compile(r''' - (((?P<y>\d\d\d\d)-)?((?P<m>\d\d?)-(?P<d>\d\d?))?)? # yyyy-mm-dd - (?P<n>\.)? # . - (((?P<H>\d?\d):(?P<M>\d\d))?(:(?P<S>\d\d))?)? # hh:mm:ss - (?P<o>.+)? # offset - ''', re.VERBOSE), serialised_re=re.compile(r''' - (\d{4})(\d\d)(\d\d)(\d\d)(\d\d)(\d\d) - ''', re.VERBOSE)): - ''' set the date to the value in spec - ''' - m = serialised_re.match(spec) - if m is not None: - # we're serialised - easy! - self.year, self.month, self.day, self.hour, self.minute, \ - self.second = map(int, m.groups()[:6]) - return - - # not serialised data, try usual format - m = date_re.match(spec) - if m is None: - raise ValueError, _('Not a date spec: [[yyyy-]mm-dd].' - '[[h]h:mm[:ss]][offset]') - - info = m.groupdict() - - # get the current date/time using the offset - y,m,d,H,M,S,x,x,x = time.gmtime(time.time()) - - # override year, month, day parts - if info['m'] is not None and info['d'] is not None: - m = int(info['m']) - d = int(info['d']) - if info['y'] is not None: y = int(info['y']) - H = M = S = 0 - - # override hour, minute, second parts - if info['H'] is not None and info['M'] is not None: - H = int(info['H']) - offset - M = int(info['M']) - S = 0 - if info['S'] is not None: S = int(info['S']) - - # now handle the adjustment of hour - ts = calendar.timegm((y,m,d,H,M,S,0,0,0)) - self.year, self.month, self.day, self.hour, self.minute, \ - self.second, x, x, x = time.gmtime(ts) - - if info.get('o', None): - self.applyInterval(Interval(info['o'])) - - def __repr__(self): - return '<Date %s>'%self.__str__() - - def local(self, offset): - """Return this date as yyyy-mm-dd.hh:mm:ss in a local time zone.""" - t = (self.year, self.month, self.day, self.hour + offset, self.minute, - self.second, 0, 0, 0) - self.year, self.month, self.day, self.hour, self.minute, \ - self.second, x, x, x = time.gmtime(calendar.timegm(t)) - - def get_tuple(self): - return (self.year, self.month, self.day, self.hour, self.minute, - self.second, 0, 0, 0) - - def serialise(self): - return '%4d%02d%02d%02d%02d%02d'%(self.year, self.month, - self.day, self.hour, self.minute, self.second) - -class Interval: - ''' - Date intervals are specified using the suffixes "y", "m", and "d". The - suffix "w" (for "week") means 7 days. Time intervals are specified in - hh:mm:ss format (the seconds may be omitted, but the hours and minutes - may not). - - "3y" means three years - "2y 1m" means two years and one month - "1m 25d" means one month and 25 days - "2w 3d" means two weeks and three days - "1d 2:50" means one day, two hours, and 50 minutes - "14:00" means 14 hours - "0:04:33" means four minutes and 33 seconds - - Example usage: - >>> Interval(" 3w 1 d 2:00") - <Interval 22d 2:00> - >>> Date(". + 2d") + Interval("- 3w") - <Date 2000-06-07.00:34:02> - - Intervals are added/subtracted in order of: - seconds, minutes, hours, years, months, days - - Calculations involving monts (eg '+2m') have no effect on days - only - days (or over/underflow from hours/mins/secs) will do that, and - days-per-month and leap years are accounted for. Leap seconds are not. - - The interval format 'syyyymmddHHMMSS' (sign, year, month, day, hour, - minute, second) is the serialisation format returned by the serialise() - method, and is accepted as an argument on instatiation. - - TODO: more examples, showing the order of addition operation - ''' - def __init__(self, spec, sign=1): - """Construct an interval given a specification.""" - if type(spec) == type(''): - self.set(spec) - else: - if len(spec) == 7: - self.sign, self.year, self.month, self.day, self.hour, \ - self.minute, self.second = spec - else: - # old, buggy spec form - self.sign = sign - self.year, self.month, self.day, self.hour, self.minute, \ - self.second = spec - - def __cmp__(self, other): - """Compare this interval to another interval.""" - if other is None: - return 1 - for attr in ('year', 'month', 'day', 'hour', 'minute', 'second'): - if not hasattr(other, attr): - return 1 - r = cmp(getattr(self, attr), getattr(other, attr)) - if r: return r - return 0 - - def __str__(self): - """Return this interval as a string.""" - sign = {1:'+', -1:'-'}[self.sign] - l = [sign] - if self.year: l.append('%sy'%self.year) - if self.month: l.append('%sm'%self.month) - if self.day: l.append('%sd'%self.day) - if self.second: - l.append('%d:%02d:%02d'%(self.hour, self.minute, self.second)) - elif self.hour or self.minute: - l.append('%d:%02d'%(self.hour, self.minute)) - return ' '.join(l) - - def set(self, spec, interval_re=re.compile(''' - \s*(?P<s>[-+])? # + or - - \s*((?P<y>\d+\s*)y)? # year - \s*((?P<m>\d+\s*)m)? # month - \s*((?P<w>\d+\s*)w)? # week - \s*((?P<d>\d+\s*)d)? # day - \s*(((?P<H>\d+):(?P<M>\d+))?(:(?P<S>\d+))?)? # time - \s*''', re.VERBOSE), serialised_re=re.compile(''' - (?P<s>[+-])(?P<y>\d{4})(?P<m>\d{2})(?P<d>\d{2}) - (?P<H>\d{2})(?P<M>\d{2})(?P<S>\d{2})''', re.VERBOSE)): - ''' set the date to the value in spec - ''' - self.year = self.month = self.week = self.day = self.hour = \ - self.minute = self.second = 0 - self.sign = 1 - m = serialised_re.match(spec) - if not m: - m = interval_re.match(spec) - if not m: - raise ValueError, _('Not an interval spec: [+-] [#y] [#m] [#w] ' - '[#d] [[[H]H:MM]:SS]') - - info = m.groupdict() - for group, attr in {'y':'year', 'm':'month', 'w':'week', 'd':'day', - 'H':'hour', 'M':'minute', 'S':'second'}.items(): - if info.get(group, None) is not None: - setattr(self, attr, int(info[group])) - - if self.week: - self.day = self.day + self.week*7 - - if info['s'] is not None: - self.sign = {'+':1, '-':-1}[info['s']] - - def __repr__(self): - return '<Interval %s>'%self.__str__() - - def pretty(self): - ''' print up the date date using one of these nice formats.. - ''' - if self.year: - if self.year == 1: - return _('1 year') - else: - return _('%(number)s years')%{'number': self.year} - elif self.month or self.day > 13: - days = (self.month * 30) + self.day - if days > 28: - if int(days/30) > 1: - s = _('%(number)s months')%{'number': int(days/30)} - else: - s = _('1 month') - else: - s = _('%(number)s weeks')%{'number': int(days/7)} - elif self.day > 7: - s = _('1 week') - elif self.day > 1: - s = _('%(number)s days')%{'number': self.day} - elif self.day == 1 or self.hour > 12: - if self.sign > 0: - return _('tomorrow') - else: - return _('yesterday') - elif self.hour > 1: - s = _('%(number)s hours')%{'number': self.hour} - elif self.hour == 1: - if self.minute < 15: - s = _('an hour') - elif self.minute/15 == 2: - s = _('1 1/2 hours') - else: - s = _('1 %(number)s/4 hours')%{'number': self.minute/15} - elif self.minute < 1: - if self.sign > 0: - return _('in a moment') - else: - return _('just now') - elif self.minute == 1: - s = _('1 minute') - elif self.minute < 15: - s = _('%(number)s minutes')%{'number': self.minute} - elif int(self.minute/15) == 2: - s = _('1/2 an hour') - else: - s = _('%(number)s/4 hour')%{'number': int(self.minute/15)} - return s - - def get_tuple(self): - return (self.sign, self.year, self.month, self.day, self.hour, - self.minute, self.second) - - def serialise(self): - return '%s%4d%02d%02d%02d%02d%02d'%(self.sign, self.year, self.month, - self.day, self.hour, self.minute, self.second) - - -def test(): - intervals = (" 3w 1 d 2:00", " + 2d", "3w") - for interval in intervals: - print '>>> Interval("%s")'%interval - print `Interval(interval)` - - dates = (".", "2000-06-25.19:34:02", ". + 2d", "1997-04-17", "01-25", - "08-13.22:13", "14:25") - for date in dates: - print '>>> Date("%s")'%date - print `Date(date)` - - sums = ((". + 2d", "3w"), (".", " 3w 1 d 2:00")) - for date, interval in sums: - print '>>> Date("%s") + Interval("%s")'%(date, interval) - print `Date(date) + Interval(interval)` - -if __name__ == '__main__': - test() - -# vim: set filetype=python ts=4 sw=4 et si
--- a/roundup/hyperdb.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,610 +0,0 @@ -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: hyperdb.py,v 1.82 2002-09-09 23:55:19 richard Exp $ - -__doc__ = """ -Hyperdatabase implementation, especially field types. -""" - -# standard python modules -import sys, os, time, re - -# roundup modules -import date, password - -# configure up the DEBUG and TRACE captures -class Sink: - def write(self, content): - pass -DEBUG = os.environ.get('HYPERDBDEBUG', '') -if DEBUG and __debug__: - if DEBUG == 'stdout': - DEBUG = sys.stdout - else: - DEBUG = open(DEBUG, 'a') -else: - DEBUG = Sink() -TRACE = os.environ.get('HYPERDBTRACE', '') -if TRACE and __debug__: - if TRACE == 'stdout': - TRACE = sys.stdout - else: - TRACE = open(TRACE, 'w') -else: - TRACE = Sink() -def traceMark(): - print >>TRACE, '**MARK', time.ctime() -del Sink - -# -# Types -# -class String: - """An object designating a String property.""" - def __init__(self, indexme='no'): - self.indexme = indexme == 'yes' - def __repr__(self): - ' more useful for dumps ' - return '<%s>'%self.__class__ - -class Password: - """An object designating a Password property.""" - def __repr__(self): - ' more useful for dumps ' - return '<%s>'%self.__class__ - -class Date: - """An object designating a Date property.""" - def __repr__(self): - ' more useful for dumps ' - return '<%s>'%self.__class__ - -class Interval: - """An object designating an Interval property.""" - def __repr__(self): - ' more useful for dumps ' - return '<%s>'%self.__class__ - -class Link: - """An object designating a Link property that links to a - node in a specified class.""" - def __init__(self, classname, do_journal='yes'): - ''' Default is to not journal link and unlink events - ''' - self.classname = classname - self.do_journal = do_journal == 'yes' - def __repr__(self): - ' more useful for dumps ' - return '<%s to "%s">'%(self.__class__, self.classname) - -class Multilink: - """An object designating a Multilink property that links - to nodes in a specified class. - - "classname" indicates the class to link to - - "do_journal" indicates whether the linked-to nodes should have - 'link' and 'unlink' events placed in their journal - """ - def __init__(self, classname, do_journal='yes'): - ''' Default is to not journal link and unlink events - ''' - self.classname = classname - self.do_journal = do_journal == 'yes' - def __repr__(self): - ' more useful for dumps ' - return '<%s to "%s">'%(self.__class__, self.classname) - -class Boolean: - """An object designating a boolean property""" - def __repr__(self): - 'more useful for dumps' - return '<%s>' % self.__class__ - -class Number: - """An object designating a numeric property""" - def __repr__(self): - 'more useful for dumps' - return '<%s>' % self.__class__ -# -# Support for splitting designators -# -class DesignatorError(ValueError): - pass -def splitDesignator(designator, dre=re.compile(r'([^\d]+)(\d+)')): - ''' Take a foo123 and return ('foo', 123) - ''' - m = dre.match(designator) - if m is None: - raise DesignatorError, '"%s" not a node designator'%designator - return m.group(1), m.group(2) - -# -# the base Database class -# -class DatabaseError(ValueError): - '''Error to be raised when there is some problem in the database code - ''' - pass -class Database: - '''A database for storing records containing flexible data types. - -This class defines a hyperdatabase storage layer, which the Classes use to -store their data. - - -Transactions ------------- -The Database should support transactions through the commit() and -rollback() methods. All other Database methods should be transaction-aware, -using data from the current transaction before looking up the database. - -An implementation must provide an override for the get() method so that the -in-database value is returned in preference to the in-transaction value. -This is necessary to determine if any values have changed during a -transaction. - - -Implementation --------------- - -All methods except __repr__ and getnode must be implemented by a -concrete backend Class. - -''' - - # flag to set on retired entries - RETIRED_FLAG = '__hyperdb_retired' - - def __init__(self, config, journaltag=None): - """Open a hyperdatabase given a specifier to some storage. - - The 'storagelocator' is obtained from config.DATABASE. - The meaning of 'storagelocator' depends on the particular - implementation of the hyperdatabase. It could be a file name, - a directory path, a socket descriptor for a connection to a - database over the network, etc. - - The 'journaltag' is a token that will be attached to the journal - entries for any edits done on the database. If 'journaltag' is - None, the database is opened in read-only mode: the Class.create(), - Class.set(), and Class.retire() methods are disabled. - """ - raise NotImplementedError - - def post_init(self): - """Called once the schema initialisation has finished.""" - raise NotImplementedError - - def __getattr__(self, classname): - """A convenient way of calling self.getclass(classname).""" - raise NotImplementedError - - def addclass(self, cl): - '''Add a Class to the hyperdatabase. - ''' - raise NotImplementedError - - def getclasses(self): - """Return a list of the names of all existing classes.""" - raise NotImplementedError - - def getclass(self, classname): - """Get the Class object representing a particular class. - - If 'classname' is not a valid class name, a KeyError is raised. - """ - raise NotImplementedError - - def clear(self): - '''Delete all database contents. - ''' - raise NotImplementedError - - def getclassdb(self, classname, mode='r'): - '''Obtain a connection to the class db that will be used for - multiple actions. - ''' - raise NotImplementedError - - def addnode(self, classname, nodeid, node): - '''Add the specified node to its class's db. - ''' - raise NotImplementedError - - def serialise(self, classname, node): - '''Copy the node contents, converting non-marshallable data into - marshallable data. - ''' - return node - - def setnode(self, classname, nodeid, node): - '''Change the specified node. - ''' - raise NotImplementedError - - def unserialise(self, classname, node): - '''Decode the marshalled node data - ''' - return node - - def getnode(self, classname, nodeid, db=None, cache=1): - '''Get a node from the database. - ''' - raise NotImplementedError - - def hasnode(self, classname, nodeid, db=None): - '''Determine if the database has a given node. - ''' - raise NotImplementedError - - def countnodes(self, classname, db=None): - '''Count the number of nodes that exist for a particular Class. - ''' - raise NotImplementedError - - def getnodeids(self, classname, db=None): - '''Retrieve all the ids of the nodes for a particular Class. - ''' - raise NotImplementedError - - def storefile(self, classname, nodeid, property, content): - '''Store the content of the file in the database. - - The property may be None, in which case the filename does not - indicate which property is being saved. - ''' - raise NotImplementedError - - def getfile(self, classname, nodeid, property): - '''Store the content of the file in the database. - ''' - raise NotImplementedError - - def addjournal(self, classname, nodeid, action, params): - ''' Journal the Action - 'action' may be: - - 'create' or 'set' -- 'params' is a dictionary of property values - 'link' or 'unlink' -- 'params' is (classname, nodeid, propname) - 'retire' -- 'params' is None - ''' - raise NotImplementedError - - def getjournal(self, classname, nodeid): - ''' get the journal for id - ''' - raise NotImplementedError - - def pack(self, pack_before): - ''' pack the database - ''' - raise NotImplementedError - - def commit(self): - ''' Commit the current transactions. - - Save all data changed since the database was opened or since the - last commit() or rollback(). - ''' - raise NotImplementedError - - def rollback(self): - ''' Reverse all actions from the current transaction. - - Undo all the changes made since the database was opened or the last - commit() or rollback() was performed. - ''' - raise NotImplementedError - -# -# The base Class class -# -class Class: - """ The handle to a particular class of nodes in a hyperdatabase. - - All methods except __repr__ and getnode must be implemented by a - concrete backend Class. - """ - - def __init__(self, db, classname, **properties): - """Create a new class with a given name and property specification. - - 'classname' must not collide with the name of an existing class, - or a ValueError is raised. The keyword arguments in 'properties' - must map names to property objects, or a TypeError is raised. - """ - raise NotImplementedError - - def __repr__(self): - '''Slightly more useful representation - ''' - return '<hyperdb.Class "%s">'%self.classname - - # Editing nodes: - - def create(self, **propvalues): - """Create a new node of this class and return its id. - - The keyword arguments in 'propvalues' map property names to values. - - The values of arguments must be acceptable for the types of their - corresponding properties or a TypeError is raised. - - If this class has a key property, it must be present and its value - must not collide with other key strings or a ValueError is raised. - - Any other properties on this class that are missing from the - 'propvalues' dictionary are set to None. - - If an id in a link or multilink property does not refer to a valid - node, an IndexError is raised. - """ - raise NotImplementedError - - _marker = [] - def get(self, nodeid, propname, default=_marker, cache=1): - """Get the value of a property on an existing node of this class. - - 'nodeid' must be the id of an existing node of this class or an - IndexError is raised. 'propname' must be the name of a property - of this class or a KeyError is raised. - - 'cache' indicates whether the transaction cache should be queried - for the node. If the node has been modified and you need to - determine what its values prior to modification are, you need to - set cache=0. - """ - raise NotImplementedError - - def getnode(self, nodeid, cache=1): - ''' Return a convenience wrapper for the node. - - 'nodeid' must be the id of an existing node of this class or an - IndexError is raised. - - 'cache' indicates whether the transaction cache should be queried - for the node. If the node has been modified and you need to - determine what its values prior to modification are, you need to - set cache=0. - ''' - return Node(self, nodeid, cache=cache) - - def set(self, nodeid, **propvalues): - """Modify a property on an existing node of this class. - - 'nodeid' must be the id of an existing node of this class or an - IndexError is raised. - - Each key in 'propvalues' must be the name of a property of this - class or a KeyError is raised. - - All values in 'propvalues' must be acceptable types for their - corresponding properties or a TypeError is raised. - - If the value of the key property is set, it must not collide with - other key strings or a ValueError is raised. - - If the value of a Link or Multilink property contains an invalid - node id, a ValueError is raised. - """ - raise NotImplementedError - - def retire(self, nodeid): - """Retire a node. - - The properties on the node remain available from the get() method, - and the node's id is never reused. - - Retired nodes are not returned by the find(), list(), or lookup() - methods, and other nodes may reuse the values of their key properties. - """ - raise NotImplementedError - - def is_retired(self, nodeid): - '''Return true if the node is rerired - ''' - raise NotImplementedError - - def destroy(self, nodeid): - """Destroy a node. - - WARNING: this method should never be used except in extremely rare - situations where there could never be links to the node being - deleted - WARNING: use retire() instead - WARNING: the properties of this node will not be available ever again - WARNING: really, use retire() instead - - Well, I think that's enough warnings. This method exists mostly to - support the session storage of the cgi interface. - - The node is completely removed from the hyperdb, including all journal - entries. It will no longer be available, and will generally break code - if there are any references to the node. - """ - - def history(self, nodeid): - """Retrieve the journal of edits on a particular node. - - 'nodeid' must be the id of an existing node of this class or an - IndexError is raised. - - The returned list contains tuples of the form - - (date, tag, action, params) - - 'date' is a Timestamp object specifying the time of the change and - 'tag' is the journaltag specified when the database was opened. - """ - raise NotImplementedError - - # Locating nodes: - def hasnode(self, nodeid): - '''Determine if the given nodeid actually exists - ''' - raise NotImplementedError - - def setkey(self, propname): - """Select a String property of this class to be the key property. - - 'propname' must be the name of a String property of this class or - None, or a TypeError is raised. The values of the key property on - all existing nodes must be unique or a ValueError is raised. - """ - raise NotImplementedError - - def getkey(self): - """Return the name of the key property for this class or None.""" - raise NotImplementedError - - def labelprop(self, default_to_id=0): - ''' Return the property name for a label for the given node. - - This method attempts to generate a consistent label for the node. - It tries the following in order: - 1. key property - 2. "name" property - 3. "title" property - 4. first property from the sorted property name list - ''' - raise NotImplementedError - - def lookup(self, keyvalue): - """Locate a particular node by its key property and return its id. - - If this class has no key property, a TypeError is raised. If the - 'keyvalue' matches one of the values for the key property among - the nodes in this class, the matching node's id is returned; - otherwise a KeyError is raised. - """ - raise NotImplementedError - - def find(self, **propspec): - """Get the ids of nodes in this class which link to the given nodes. - - 'propspec' consists of keyword args propname={nodeid:1,} - 'propname' must be the name of a property in this class, or a - KeyError is raised. That property must be a Link or Multilink - property, or a TypeError is raised. - - Any node in this class whose 'propname' property links to any of the - nodeids will be returned. Used by the full text indexing, which knows - that "foo" occurs in msg1, msg3 and file7, so we have hits on these - issues: - - db.issue.find(messages={'1':1,'3':1}, files={'7':1}) - """ - raise NotImplementedError - - def filter(self, search_matches, filterspec, sort, group, - num_re = re.compile('^\d+$')): - ''' Return a list of the ids of the active nodes in this class that - match the 'filter' spec, sorted by the group spec and then the - sort spec - ''' - raise NotImplementedError - - def count(self): - """Get the number of nodes in this class. - - If the returned integer is 'numnodes', the ids of all the nodes - in this class run from 1 to numnodes, and numnodes+1 will be the - id of the next node to be created in this class. - """ - raise NotImplementedError - - # Manipulating properties: - def getprops(self, protected=1): - """Return a dictionary mapping property names to property objects. - If the "protected" flag is true, we include protected properties - - those which may not be modified. - """ - raise NotImplementedError - - def addprop(self, **properties): - """Add properties to this class. - - The keyword arguments in 'properties' must map names to property - objects, or a TypeError is raised. None of the keys in 'properties' - may collide with the names of existing properties, or a ValueError - is raised before any properties have been added. - """ - raise NotImplementedError - - def index(self, nodeid): - '''Add (or refresh) the node to search indexes - ''' - raise NotImplementedError - -class Node: - ''' A convenience wrapper for the given node - ''' - def __init__(self, cl, nodeid, cache=1): - self.__dict__['cl'] = cl - self.__dict__['nodeid'] = nodeid - self.__dict__['cache'] = cache - def keys(self, protected=1): - return self.cl.getprops(protected=protected).keys() - def values(self, protected=1): - l = [] - for name in self.cl.getprops(protected=protected).keys(): - l.append(self.cl.get(self.nodeid, name, cache=self.cache)) - return l - def items(self, protected=1): - l = [] - for name in self.cl.getprops(protected=protected).keys(): - l.append((name, self.cl.get(self.nodeid, name, cache=self.cache))) - return l - def has_key(self, name): - return self.cl.getprops().has_key(name) - def __getattr__(self, name): - if self.__dict__.has_key(name): - return self.__dict__[name] - try: - return self.cl.get(self.nodeid, name, cache=self.cache) - except KeyError, value: - # we trap this but re-raise it as AttributeError - all other - # exceptions should pass through untrapped - pass - # nope, no such attribute - raise AttributeError, str(value) - def __getitem__(self, name): - return self.cl.get(self.nodeid, name, cache=self.cache) - def __setattr__(self, name, value): - try: - return self.cl.set(self.nodeid, **{name: value}) - except KeyError, value: - raise AttributeError, str(value) - def __setitem__(self, name, value): - self.cl.set(self.nodeid, **{name: value}) - def history(self): - return self.cl.history(self.nodeid) - def retire(self): - return self.cl.retire(self.nodeid) - - -def Choice(name, db, *options): - '''Quick helper to create a simple class with choices - ''' - cl = Class(db, name, name=String(), order=String()) - for i in range(len(options)): - cl.create(name=options[i], order=i) - return hyperdb.Link(name) - -# vim: set filetype=python ts=4 sw=4 et si
--- a/roundup/mailgw.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,944 +0,0 @@ -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# - -__doc__ = ''' -An e-mail gateway for Roundup. - -Incoming messages are examined for multiple parts: - . In a multipart/mixed message or part, each subpart is extracted and - examined. The text/plain subparts are assembled to form the textual - body of the message, to be stored in the file associated with a "msg" - class node. Any parts of other types are each stored in separate files - and given "file" class nodes that are linked to the "msg" node. - . In a multipart/alternative message or part, we look for a text/plain - subpart and ignore the other parts. - -Summary -------- -The "summary" property on message nodes is taken from the first non-quoting -section in the message body. The message body is divided into sections by -blank lines. Sections where the second and all subsequent lines begin with -a ">" or "|" character are considered "quoting sections". The first line of -the first non-quoting section becomes the summary of the message. - -Addresses ---------- -All of the addresses in the To: and Cc: headers of the incoming message are -looked up among the user nodes, and the corresponding users are placed in -the "recipients" property on the new "msg" node. The address in the From: -header similarly determines the "author" property of the new "msg" -node. The default handling for addresses that don't have corresponding -users is to create new users with no passwords and a username equal to the -address. (The web interface does not permit logins for users with no -passwords.) If we prefer to reject mail from outside sources, we can simply -register an auditor on the "user" class that prevents the creation of user -nodes with no passwords. - -Actions -------- -The subject line of the incoming message is examined to determine whether -the message is an attempt to create a new item or to discuss an existing -item. A designator enclosed in square brackets is sought as the first thing -on the subject line (after skipping any "Fwd:" or "Re:" prefixes). - -If an item designator (class name and id number) is found there, the newly -created "msg" node is added to the "messages" property for that item, and -any new "file" nodes are added to the "files" property for the item. - -If just an item class name is found there, we attempt to create a new item -of that class with its "messages" property initialized to contain the new -"msg" node and its "files" property initialized to contain any new "file" -nodes. - -Triggers --------- -Both cases may trigger detectors (in the first case we are calling the -set() method to add the message to the item's spool; in the second case we -are calling the create() method to create a new node). If an auditor raises -an exception, the original message is bounced back to the sender with the -explanatory message given in the exception. - -$Id: mailgw.py,v 1.92 2002-09-26 03:03:18 richard Exp $ -''' - -import string, re, os, mimetools, cStringIO, smtplib, socket, binascii, quopri -import time, random, sys -import traceback, MimeWriter -import hyperdb, date, password - -SENDMAILDEBUG = os.environ.get('SENDMAILDEBUG', '') - -class MailGWError(ValueError): - pass - -class MailUsageError(ValueError): - pass - -class MailUsageHelp(Exception): - pass - -class Unauthorized(Exception): - """ Access denied """ - -def initialiseSecurity(security): - ''' Create some Permissions and Roles on the security object - - This function is directly invoked by security.Security.__init__() - as a part of the Security object instantiation. - ''' - security.addPermission(name="Email Registration", - description="Anonymous may register through e-mail") - p = security.addPermission(name="Email Access", - description="User may use the email interface") - security.addPermissionToRole('Admin', p) - -class Message(mimetools.Message): - ''' subclass mimetools.Message so we can retrieve the parts of the - message... - ''' - def getPart(self): - ''' Get a single part of a multipart message and return it as a new - Message instance. - ''' - boundary = self.getparam('boundary') - mid, end = '--'+boundary, '--'+boundary+'--' - s = cStringIO.StringIO() - while 1: - line = self.fp.readline() - if not line: - break - if line.strip() in (mid, end): - break - s.write(line) - if not s.getvalue().strip(): - return None - s.seek(0) - return Message(s) - -subject_re = re.compile(r'(?P<refwd>\s*\W?\s*(fwd|re|aw)\s*\W?\s*)*' - r'\s*(?P<quote>")?(\[(?P<classname>[^\d\s]+)(?P<nodeid>\d+)?\])?' - r'\s*(?P<title>[^[]+)?"?(\[(?P<args>.+?)\])?', re.I) - -class MailGW: - def __init__(self, instance, db): - self.instance = instance - self.db = db - - # should we trap exceptions (normal usage) or pass them through - # (for testing) - self.trapExceptions = 1 - - def do_pipe(self): - ''' Read a message from standard input and pass it to the mail handler. - ''' - self.main(sys.stdin) - return 0 - - def do_mailbox(self, filename): - ''' Read a series of messages from the specified unix mailbox file and - pass each to the mail handler. - ''' - # open the spool file and lock it - import fcntl, FCNTL - f = open(filename, 'r+') - fcntl.flock(f.fileno(), FCNTL.LOCK_EX) - - # handle and clear the mailbox - try: - from mailbox import UnixMailbox - mailbox = UnixMailbox(f, factory=Message) - # grab one message - message = mailbox.next() - while message: - # handle this message - self.handle_Message(message) - message = mailbox.next() - # nuke the file contents - os.ftruncate(f.fileno(), 0) - except: - import traceback - traceback.print_exc() - return 1 - fcntl.flock(f.fileno(), FCNTL.LOCK_UN) - return 0 - - def do_pop(self, server, user='', password=''): - '''Read a series of messages from the specified POP server. - ''' - import getpass, poplib, socket - try: - if not user: - user = raw_input(_('User: ')) - if not password: - password = getpass.getpass() - except (KeyboardInterrupt, EOFError): - # Ctrl C or D maybe also Ctrl Z under Windows. - print "\nAborted by user." - return 1 - - # open a connection to the server and retrieve all messages - try: - server = poplib.POP3(server) - except socket.error, message: - print "POP server error:", message - return 1 - server.user(user) - server.pass_(password) - numMessages = len(server.list()[1]) - for i in range(1, numMessages+1): - # retr: returns - # [ pop response e.g. '+OK 459 octets', - # [ array of message lines ], - # number of octets ] - lines = server.retr(i)[1] - s = cStringIO.StringIO('\n'.join(lines)) - s.seek(0) - self.handle_Message(Message(s)) - # delete the message - server.dele(i) - - # quit the server to commit changes. - server.quit() - return 0 - - def main(self, fp): - ''' fp - the file from which to read the Message. - ''' - return self.handle_Message(Message(fp)) - - def handle_Message(self, message): - '''Handle an RFC822 Message - - Handle the Message object by calling handle_message() and then cope - with any errors raised by handle_message. - This method's job is to make that call and handle any - errors in a sane manner. It should be replaced if you wish to - handle errors in a different manner. - ''' - # in some rare cases, a particularly stuffed-up e-mail will make - # its way into here... try to handle it gracefully - sendto = message.getaddrlist('from') - if sendto: - if not self.trapExceptions: - return self.handle_message(message) - try: - return self.handle_message(message) - except MailUsageHelp: - # bounce the message back to the sender with the usage message - fulldoc = '\n'.join(string.split(__doc__, '\n')[2:]) - sendto = [sendto[0][1]] - m = [''] - m.append('\n\nMail Gateway Help\n=================') - m.append(fulldoc) - m = self.bounce_message(message, sendto, m, - subject="Mail Gateway Help") - except MailUsageError, value: - # bounce the message back to the sender with the usage message - fulldoc = '\n'.join(string.split(__doc__, '\n')[2:]) - sendto = [sendto[0][1]] - m = [''] - m.append(str(value)) - m.append('\n\nMail Gateway Help\n=================') - m.append(fulldoc) - m = self.bounce_message(message, sendto, m) - except Unauthorized, value: - # just inform the user that he is not authorized - sendto = [sendto[0][1]] - m = [''] - m.append(str(value)) - m = self.bounce_message(message, sendto, m) - except: - # bounce the message back to the sender with the error message - sendto = [sendto[0][1], self.instance.config.ADMIN_EMAIL] - m = [''] - m.append('An unexpected error occurred during the processing') - m.append('of your message. The tracker administrator is being') - m.append('notified.\n') - m.append('---- traceback of failure ----') - s = cStringIO.StringIO() - import traceback - traceback.print_exc(None, s) - m.append(s.getvalue()) - m = self.bounce_message(message, sendto, m) - else: - # very bad-looking message - we don't even know who sent it - sendto = [self.instance.config.ADMIN_EMAIL] - m = ['Subject: badly formed message from mail gateway'] - m.append('') - m.append('The mail gateway retrieved a message which has no From:') - m.append('line, indicating that it is corrupt. Please check your') - m.append('mail gateway source. Failed message is attached.') - m.append('') - m = self.bounce_message(message, sendto, m, - subject='Badly formed message from mail gateway') - - # now send the message - if SENDMAILDEBUG: - open(SENDMAILDEBUG, 'w').write('From: %s\nTo: %s\n%s\n'%( - self.instance.config.ADMIN_EMAIL, ', '.join(sendto), - m.getvalue())) - else: - try: - smtp = smtplib.SMTP(self.instance.config.MAILHOST) - smtp.sendmail(self.instance.config.ADMIN_EMAIL, sendto, - m.getvalue()) - except socket.error, value: - raise MailGWError, "Couldn't send error email: "\ - "mailhost %s"%value - except smtplib.SMTPException, value: - raise MailGWError, "Couldn't send error email: %s"%value - - def bounce_message(self, message, sendto, error, - subject='Failed issue tracker submission'): - ''' create a message that explains the reason for the failed - issue submission to the author and attach the original - message. - ''' - msg = cStringIO.StringIO() - writer = MimeWriter.MimeWriter(msg) - writer.addheader('Subject', subject) - writer.addheader('From', '%s <%s>'% (self.instance.config.TRACKER_NAME, - self.instance.config.TRACKER_EMAIL)) - writer.addheader('To', ','.join(sendto)) - writer.addheader('MIME-Version', '1.0') - part = writer.startmultipartbody('mixed') - part = writer.nextpart() - body = part.startbody('text/plain') - body.write('\n'.join(error)) - - # reconstruct the original message - m = cStringIO.StringIO() - w = MimeWriter.MimeWriter(m) - # default the content_type, just in case... - content_type = 'text/plain' - # add the headers except the content-type - for header in message.headers: - header_name = header.split(':')[0] - if header_name.lower() == 'content-type': - content_type = message.getheader(header_name) - elif message.getheader(header_name): - w.addheader(header_name, message.getheader(header_name)) - # now attach the message body - body = w.startbody(content_type) - try: - message.rewindbody() - except IOError: - body.write("*** couldn't include message body: read from pipe ***") - else: - body.write(message.fp.read()) - - # attach the original message to the returned message - part = writer.nextpart() - part.addheader('Content-Disposition','attachment') - part.addheader('Content-Description','Message you sent') - part.addheader('Content-Transfer-Encoding', '7bit') - body = part.startbody('message/rfc822') - body.write(m.getvalue()) - - writer.lastpart() - return msg - - def get_part_data_decoded(self,part): - encoding = part.getencoding() - data = None - if encoding == 'base64': - # BUG: is base64 really used for text encoding or - # are we inserting zip files here. - data = binascii.a2b_base64(part.fp.read()) - elif encoding == 'quoted-printable': - # the quopri module wants to work with files - decoded = cStringIO.StringIO() - quopri.decode(part.fp, decoded) - data = decoded.getvalue() - elif encoding == 'uuencoded': - data = binascii.a2b_uu(part.fp.read()) - else: - # take it as text - data = part.fp.read() - return data - - def handle_message(self, message): - ''' message - a Message instance - - Parse the message as per the module docstring. - ''' - # handle the subject line - subject = message.getheader('subject', '') - - if subject.strip() == 'help': - raise MailUsageHelp - - m = subject_re.match(subject) - - # check for well-formed subject line - if m: - # get the classname - classname = m.group('classname') - if classname is None: - # no classname, fallback on the default - if hasattr(self.instance.config, 'MAIL_DEFAULT_CLASS') and \ - self.instance.config.MAIL_DEFAULT_CLASS: - classname = self.instance.config.MAIL_DEFAULT_CLASS - else: - # fail - m = None - - if not m: - raise MailUsageError, ''' -The message you sent to roundup did not contain a properly formed subject -line. The subject must contain a class name or designator to indicate the -"topic" of the message. For example: - Subject: [issue] This is a new issue - - this will create a new issue in the tracker with the title "This is - a new issue". - Subject: [issue1234] This is a followup to issue 1234 - - this will append the message's contents to the existing issue 1234 - in the tracker. - -Subject was: "%s" -'''%subject - - # get the class - try: - cl = self.db.getclass(classname) - except KeyError: - raise MailUsageError, ''' -The class name you identified in the subject line ("%s") does not exist in the -database. - -Valid class names are: %s -Subject was: "%s" -'''%(classname, ', '.join(self.db.getclasses()), subject) - - # get the optional nodeid - nodeid = m.group('nodeid') - - # title is optional too - title = m.group('title') - if title: - title = title.strip() - else: - title = '' - - # strip off the quotes that dumb emailers put around the subject, like - # Re: "[issue1] bla blah" - if m.group('quote') and title.endswith('"'): - title = title[:-1] - - # but we do need either a title or a nodeid... - if nodeid is None and not title: - raise MailUsageError, ''' -I cannot match your message to a node in the database - you need to either -supply a full node identifier (with number, eg "[issue123]" or keep the -previous subject title intact so I can match that. - -Subject was: "%s" -'''%subject - - # If there's no nodeid, check to see if this is a followup and - # maybe someone's responded to the initial mail that created an - # entry. Try to find the matching nodes with the same title, and - # use the _last_ one matched (since that'll _usually_ be the most - # recent...) - if nodeid is None and m.group('refwd'): - l = cl.stringFind(title=title) - if l: - nodeid = l[-1] - - # if a nodeid was specified, make sure it's valid - if nodeid is not None and not cl.hasnode(nodeid): - raise MailUsageError, ''' -The node specified by the designator in the subject of your message ("%s") -does not exist. - -Subject was: "%s" -'''%(nodeid, subject) - - # - # handle the users - # - # Don't create users if anonymous isn't allowed to register - create = 1 - anonid = self.db.user.lookup('anonymous') - if not self.db.security.hasPermission('Email Registration', anonid): - create = 0 - - # ok, now figure out who the author is - create a new user if the - # "create" flag is true - author = uidFromAddress(self.db, message.getaddrlist('from')[0], - create=create) - - # no author? means we're not author - if not author: - raise Unauthorized, ''' -You are not a registered user. - -Unknown address: %s -'''%message.getaddrlist('from')[0][1] - - # make sure the author has permission to use the email interface - if not self.db.security.hasPermission('Email Access', author): - raise Unauthorized, 'You are not permitted to access this tracker.' - - # make sure they're allowed to edit this class of information - if not self.db.security.hasPermission('Edit', author, classname): - raise Unauthorized, 'You are not permitted to edit %s.'%classname - - # the author may have been created - make sure the change is - # committed before we reopen the database - self.db.commit() - - # reopen the database as the author - username = self.db.user.get(author, 'username') - self.db.close() - self.db = self.instance.open(username) - - # re-get the class with the new database connection - cl = self.db.getclass(classname) - - # now update the recipients list - recipients = [] - tracker_email = self.instance.config.TRACKER_EMAIL.lower() - for recipient in message.getaddrlist('to') + message.getaddrlist('cc'): - r = recipient[1].strip().lower() - if r == tracker_email or not r: - continue - - # look up the recipient - create if necessary (and we're - # allowed to) - recipient = uidFromAddress(self.db, recipient, create) - - # if all's well, add the recipient to the list - if recipient: - recipients.append(recipient) - - # - # extract the args - # - subject_args = m.group('args') - - # - # handle the subject argument list - # - # figure what the properties of this Class are - properties = cl.getprops() - props = {} - args = m.group('args') - if args: - errors = [] - for prop in string.split(args, ';'): - # extract the property name and value - try: - propname, value = prop.split('=') - except ValueError, message: - errors.append('not of form [arg=value,' - 'value,...;arg=value,value...]') - break - - # ensure it's a valid property name - propname = propname.strip() - try: - proptype = properties[propname] - except KeyError: - errors.append('refers to an invalid property: ' - '"%s"'%propname) - continue - - # convert the string value to a real property value - if isinstance(proptype, hyperdb.String): - props[propname] = value.strip() - if isinstance(proptype, hyperdb.Password): - props[propname] = password.Password(value.strip()) - elif isinstance(proptype, hyperdb.Date): - try: - props[propname] = date.Date(value.strip()) - except ValueError, message: - errors.append('contains an invalid date for ' - '%s.'%propname) - elif isinstance(proptype, hyperdb.Interval): - try: - props[propname] = date.Interval(value) - except ValueError, message: - errors.append('contains an invalid date interval' - 'for %s.'%propname) - elif isinstance(proptype, hyperdb.Link): - linkcl = self.db.classes[proptype.classname] - propkey = linkcl.labelprop(default_to_id=1) - try: - props[propname] = linkcl.lookup(value) - except KeyError, message: - errors.append('"%s" is not a value for %s.'%(value, - propname)) - elif isinstance(proptype, hyperdb.Multilink): - # get the linked class - linkcl = self.db.classes[proptype.classname] - propkey = linkcl.labelprop(default_to_id=1) - if nodeid: - curvalue = cl.get(nodeid, propname) - else: - curvalue = [] - - # handle each add/remove in turn - # keep an extra list for all items that are - # definitely in the new list (in case of e.g. - # <propname>=A,+B, which should replace the old - # list with A,B) - set = 0 - newvalue = [] - for item in value.split(','): - item = item.strip() - - # handle +/- - remove = 0 - if item.startswith('-'): - remove = 1 - item = item[1:] - elif item.startswith('+'): - item = item[1:] - else: - set = 1 - - # look up the value - try: - item = linkcl.lookup(item) - except KeyError, message: - errors.append('"%s" is not a value for %s.'%(item, - propname)) - continue - - # perform the add/remove - if remove: - try: - curvalue.remove(item) - except ValueError: - errors.append('"%s" is not currently in ' - 'for %s.'%(item, propname)) - continue - else: - newvalue.append(item) - if item not in curvalue: - curvalue.append(item) - - # that's it, set the new Multilink property value, - # or overwrite it completely - if set: - props[propname] = newvalue - else: - props[propname] = curvalue - elif isinstance(proptype, hyperdb.Boolean): - value = value.strip() - props[propname] = value.lower() in ('yes', 'true', 'on', '1') - elif isinstance(proptype, hyperdb.Number): - value = value.strip() - props[propname] = int(value) - - # handle any errors parsing the argument list - if errors: - errors = '\n- '.join(errors) - raise MailUsageError, ''' -There were problems handling your subject line argument list: -- %s - -Subject was: "%s" -'''%(errors, subject) - - # - # handle message-id and in-reply-to - # - messageid = message.getheader('message-id') - inreplyto = message.getheader('in-reply-to') or '' - # generate a messageid if there isn't one - if not messageid: - messageid = "<%s.%s.%s%s@%s>"%(time.time(), random.random(), - classname, nodeid, self.instance.config.MAIL_DOMAIN) - - # - # now handle the body - find the message - # - content_type = message.gettype() - attachments = [] - # General multipart handling: - # Take the first text/plain part, anything else is considered an - # attachment. - # multipart/mixed: multiple "unrelated" parts. - # multipart/signed (rfc 1847): - # The control information is carried in the second of the two - # required body parts. - # ACTION: Default, so if content is text/plain we get it. - # multipart/encrypted (rfc 1847): - # The control information is carried in the first of the two - # required body parts. - # ACTION: Not handleable as the content is encrypted. - # multipart/related (rfc 1872, 2112, 2387): - # The Multipart/Related content-type addresses the MIME - # representation of compound objects. - # ACTION: Default. If we are lucky there is a text/plain. - # TODO: One should use the start part and look for an Alternative - # that is text/plain. - # multipart/Alternative (rfc 1872, 1892): - # only in "related" ? - # multipart/report (rfc 1892): - # e.g. mail system delivery status reports. - # ACTION: Default. Could be ignored or used for Delivery Notification - # flagging. - # multipart/form-data: - # For web forms only. - if content_type == 'multipart/mixed': - # skip over the intro to the first boundary - part = message.getPart() - content = None - while 1: - # get the next part - part = message.getPart() - if part is None: - break - # parse it - subtype = part.gettype() - if subtype == 'text/plain' and not content: - # The first text/plain part is the message content. - content = self.get_part_data_decoded(part) - elif subtype == 'message/rfc822': - # handle message/rfc822 specially - the name should be - # the subject of the actual e-mail embedded here - i = part.fp.tell() - mailmess = Message(part.fp) - name = mailmess.getheader('subject') - part.fp.seek(i) - attachments.append((name, 'message/rfc822', part.fp.read())) - else: - # try name on Content-Type - name = part.getparam('name') - # this is just an attachment - data = self.get_part_data_decoded(part) - attachments.append((name, part.gettype(), data)) - if content is None: - raise MailUsageError, ''' -Roundup requires the submission to be plain text. The message parser could -not find a text/plain part to use. -''' - - elif content_type[:10] == 'multipart/': - # skip over the intro to the first boundary - message.getPart() - content = None - while 1: - # get the next part - part = message.getPart() - if part is None: - break - # parse it - if part.gettype() == 'text/plain' and not content: - content = self.get_part_data_decoded(part) - if content is None: - raise MailUsageError, ''' -Roundup requires the submission to be plain text. The message parser could -not find a text/plain part to use. -''' - - elif content_type != 'text/plain': - raise MailUsageError, ''' -Roundup requires the submission to be plain text. The message parser could -not find a text/plain part to use. -''' - - else: - content = self.get_part_data_decoded(message) - - # figure how much we should muck around with the email body - keep_citations = getattr(self.instance, 'EMAIL_KEEP_QUOTED_TEXT', - 'no') == 'yes' - keep_body = getattr(self.instance, 'EMAIL_LEAVE_BODY_UNCHANGED', - 'no') == 'yes' - - # parse the body of the message, stripping out bits as appropriate - summary, content = parseContent(content, keep_citations, - keep_body) - - # - # handle the attachments - # - files = [] - for (name, mime_type, data) in attachments: - if not name: - name = "unnamed" - files.append(self.db.file.create(type=mime_type, name=name, - content=data)) - - # - # create the message if there's a message body (content) - # - if content: - message_id = self.db.msg.create(author=author, - recipients=recipients, date=date.Date('.'), summary=summary, - content=content, files=files, messageid=messageid, - inreplyto=inreplyto) - - # attach the message to the node - if nodeid: - # add the message to the node's list - messages = cl.get(nodeid, 'messages') - messages.append(message_id) - props['messages'] = messages - else: - # pre-load the messages list - props['messages'] = [message_id] - - # set the title to the subject - if properties.has_key('title') and not props.has_key('title'): - props['title'] = title - - # - # perform the node change / create - # - try: - if nodeid: - cl.set(nodeid, **props) - else: - nodeid = cl.create(**props) - except (TypeError, IndexError, ValueError), message: - raise MailUsageError, ''' -There was a problem with the message you sent: - %s -'''%message - - # commit the changes to the DB - self.db.commit() - - return nodeid - -def extractUserFromList(userClass, users): - '''Given a list of users, try to extract the first non-anonymous user - and return that user, otherwise return None - ''' - if len(users) > 1: - for user in users: - # make sure we don't match the anonymous or admin user - if userClass.get(user, 'username') in ('admin', 'anonymous'): - continue - # first valid match will do - return user - # well, I guess we have no choice - return user[0] - elif users: - return users[0] - return None - -def uidFromAddress(db, address, create=1): - ''' address is from the rfc822 module, and therefore is (name, addr) - - user is created if they don't exist in the db already - ''' - (realname, address) = address - - # try a straight match of the address - user = extractUserFromList(db.user, db.user.stringFind(address=address)) - if user is not None: return user - - # try the user alternate addresses if possible - props = db.user.getprops() - if props.has_key('alternate_addresses'): - users = db.user.filter(None, {'alternate_addresses': address}, - [], []) - user = extractUserFromList(db.user, users) - if user is not None: return user - - # try to match the username to the address (for local - # submissions where the address is empty) - user = extractUserFromList(db.user, db.user.stringFind(username=address)) - - # couldn't match address or username, so create a new user - if create: - return db.user.create(username=address, address=address, - realname=realname, roles=db.config.NEW_EMAIL_USER_ROLES) - else: - return 0 - -def parseContent(content, keep_citations, keep_body, - blank_line=re.compile(r'[\r\n]+\s*[\r\n]+'), - eol=re.compile(r'[\r\n]+'), - signature=re.compile(r'^[>|\s]*[-_]+\s*$'), - original_message=re.compile(r'^[>|\s]*-----Original Message-----$')): - ''' The message body is divided into sections by blank lines. - Sections where the second and all subsequent lines begin with a ">" - or "|" character are considered "quoting sections". The first line of - the first non-quoting section becomes the summary of the message. - - If keep_citations is true, then we keep the "quoting sections" in the - content. - If keep_body is true, we even keep the signature sections. - ''' - # strip off leading carriage-returns / newlines - i = 0 - for i in range(len(content)): - if content[i] not in '\r\n': - break - if i > 0: - sections = blank_line.split(content[i:]) - else: - sections = blank_line.split(content) - - # extract out the summary from the message - summary = '' - l = [] - for section in sections: - #section = section.strip() - if not section: - continue - lines = eol.split(section) - if (lines[0] and lines[0][0] in '>|') or (len(lines) > 1 and - lines[1] and lines[1][0] in '>|'): - # see if there's a response somewhere inside this section (ie. - # no blank line between quoted message and response) - for line in lines[1:]: - if line and line[0] not in '>|': - break - else: - # we keep quoted bits if specified in the config - if keep_citations: - l.append(section) - continue - # keep this section - it has reponse stuff in it - if not summary: - # and while we're at it, use the first non-quoted bit as - # our summary - summary = line - lines = lines[lines.index(line):] - section = '\n'.join(lines) - - if not summary: - # if we don't have our summary yet use the first line of this - # section - summary = lines[0] - elif signature.match(lines[0]) and 2 <= len(lines) <= 10: - # lose any signature - break - elif original_message.match(lines[0]): - # ditch the stupid Outlook quoting of the entire original message - break - - # and add the section to the output - l.append(section) - - # Now reconstitute the message content minus the bits we don't care - # about. - if not keep_body: - content = '\n\n'.join(l) - - return summary, content - -# vim: set filetype=python ts=4 sw=4 et si
--- a/roundup/password.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,143 +0,0 @@ -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: password.py,v 1.7 2002-09-26 13:38:35 gmcm Exp $ - -__doc__ = """ -Password handling (encoding, decoding). -""" - -import sha, re, string -try: - import crypt -except: - crypt = None - pass - -def encodePassword(plaintext, scheme, other=None): - '''Encrypt the plaintext password. - ''' - if scheme == 'SHA': - s = sha.sha(plaintext).hexdigest() - elif scheme == 'crypt' and crypt is not None: - if other is not None: - salt = other[:2] - else: - saltchars = './0123456789'+string.letters - salt = random.choice(saltchars) + random.choice(saltchars) - s = crypt.crypt(plaintext, salt) - elif scheme == 'plaintext': - s = plaintext - else: - raise ValueError, 'Unknown encryption scheme "%s"'%scheme - return s - -class Password: - '''The class encapsulates a Password property type value in the database. - - The encoding of the password is one if None, 'SHA' or 'plaintext'. The - encodePassword function is used to actually encode the password from - plaintext. The None encoding is used in legacy databases where no - encoding scheme is identified. - - The scheme is stored with the encoded data in the database: - {scheme}data - - Example usage: - >>> p = Password('sekrit') - >>> p == 'sekrit' - 1 - >>> p != 'not sekrit' - 1 - >>> 'sekrit' == p - 1 - >>> 'not sekrit' != p - 1 - ''' - - default_scheme = 'SHA' # new encryptions use this scheme - pwre = re.compile(r'{(\w+)}(.+)') - - def __init__(self, plaintext=None, scheme=None): - '''Call setPassword if plaintext is not None.''' - if scheme is None: - scheme = self.default_scheme - if plaintext is not None: - self.password = encodePassword(plaintext, self.default_scheme) - self.scheme = self.default_scheme - else: - self.password = None - self.scheme = self.default_scheme - - def unpack(self, encrypted): - '''Set the password info from the scheme:<encryted info> string - (the inverse of __str__) - ''' - m = self.pwre.match(encrypted) - if m: - self.scheme = m.group(1) - self.password = m.group(2) - else: - # currently plaintext - encrypt - self.password = encodePassword(encrypted, self.default_scheme) - self.scheme = self.default_scheme - - def setPassword(self, plaintext, scheme=None): - '''Sets encrypts plaintext.''' - if scheme is None: - scheme = self.default_scheme - self.password = encodePassword(plaintext, scheme) - - def __cmp__(self, other): - '''Compare this password against another password.''' - # check to see if we're comparing instances - if isinstance(other, Password): - if self.scheme != other.scheme: - return cmp(self.scheme, other.scheme) - return cmp(self.password, other.password) - - # assume password is plaintext - if self.password is None: - raise ValueError, 'Password not set' - return cmp(self.password, encodePassword(other, self.scheme, - self.password)) - - def __str__(self): - '''Stringify the encrypted password for database storage.''' - if self.password is None: - raise ValueError, 'Password not set' - return '{%s}%s'%(self.scheme, self.password) - -def test(): - # SHA - p = Password('sekrit') - assert p == 'sekrit' - assert p != 'not sekrit' - assert 'sekrit' == p - assert 'not sekrit' != p - - # crypt - p = Password('sekrit', 'crypt') - assert p == 'sekrit' - assert p != 'not sekrit' - assert 'sekrit' == p - assert 'not sekrit' != p - -if __name__ == '__main__': - test() - -# vim: set filetype=python ts=4 sw=4 et si
--- a/roundup/roundupdb.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,433 +0,0 @@ -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: roundupdb.py,v 1.69 2002-09-20 01:20:31 richard Exp $ - -__doc__ = """ -Extending hyperdb with types specific to issue-tracking. -""" - -import re, os, smtplib, socket, time, random -import MimeWriter, cStringIO -import base64, quopri, mimetypes -# if available, use the 'email' module, otherwise fallback to 'rfc822' -try : - from email.Utils import dump_address_pair as straddr -except ImportError : - from rfc822 import dump_address_pair as straddr - -import hyperdb - -# set to indicate to roundup not to actually _send_ email -# this var must contain a file to write the mail to -SENDMAILDEBUG = os.environ.get('SENDMAILDEBUG', '') - -class Database: - def getuid(self): - """Return the id of the "user" node associated with the user - that owns this connection to the hyperdatabase.""" - return self.user.lookup(self.journaltag) - -class MessageSendError(RuntimeError): - pass - -class DetectorError(RuntimeError): - ''' Raised by detectors that want to indicate that something's amiss - ''' - pass - -# deviation from spec - was called IssueClass -class IssueClass: - """ This class is intended to be mixed-in with a hyperdb backend - implementation. The backend should provide a mechanism that - enforces the title, messages, files, nosy and superseder - properties: - properties['title'] = hyperdb.String(indexme='yes') - properties['messages'] = hyperdb.Multilink("msg") - properties['files'] = hyperdb.Multilink("file") - properties['nosy'] = hyperdb.Multilink("user") - properties['superseder'] = hyperdb.Multilink(classname) - """ - - # New methods: - def addmessage(self, nodeid, summary, text): - """Add a message to an issue's mail spool. - - A new "msg" node is constructed using the current date, the user that - owns the database connection as the author, and the specified summary - text. - - The "files" and "recipients" fields are left empty. - - The given text is saved as the body of the message and the node is - appended to the "messages" field of the specified issue. - """ - - def nosymessage(self, nodeid, msgid, oldvalues): - """Send a message to the members of an issue's nosy list. - - The message is sent only to users on the nosy list who are not - already on the "recipients" list for the message. - - These users are then added to the message's "recipients" list. - """ - users = self.db.user - messages = self.db.msg - - # figure the recipient ids - sendto = [] - r = {} - recipients = messages.get(msgid, 'recipients') - for recipid in messages.get(msgid, 'recipients'): - r[recipid] = 1 - - # figure the author's id, and indicate they've received the message - authid = messages.get(msgid, 'author') - - # possibly send the message to the author, as long as they aren't - # anonymous - if (self.db.config.MESSAGES_TO_AUTHOR == 'yes' and - users.get(authid, 'username') != 'anonymous'): - sendto.append(authid) - r[authid] = 1 - - # now figure the nosy people who weren't recipients - nosy = self.get(nodeid, 'nosy') - for nosyid in nosy: - # Don't send nosy mail to the anonymous user (that user - # shouldn't appear in the nosy list, but just in case they - # do...) - if users.get(nosyid, 'username') == 'anonymous': - continue - # make sure they haven't seen the message already - if not r.has_key(nosyid): - # send it to them - sendto.append(nosyid) - recipients.append(nosyid) - - # generate a change note - if oldvalues: - note = self.generateChangeNote(nodeid, oldvalues) - else: - note = self.generateCreateNote(nodeid) - - # we have new recipients - if sendto: - # map userids to addresses - sendto = [users.get(i, 'address') for i in sendto] - - # update the message's recipients list - messages.set(msgid, recipients=recipients) - - # send the message - self.send_message(nodeid, msgid, note, sendto) - - # backwards compatibility - don't remove - sendmessage = nosymessage - - def send_message(self, nodeid, msgid, note, sendto): - '''Actually send the nominated message from this node to the sendto - recipients, with the note appended. - ''' - users = self.db.user - messages = self.db.msg - files = self.db.file - - # determine the messageid and inreplyto of the message - inreplyto = messages.get(msgid, 'inreplyto') - messageid = messages.get(msgid, 'messageid') - - # make up a messageid if there isn't one (web edit) - if not messageid: - # this is an old message that didn't get a messageid, so - # create one - messageid = "<%s.%s.%s%s@%s>"%(time.time(), random.random(), - self.classname, nodeid, self.db.config.MAIL_DOMAIN) - messages.set(msgid, messageid=messageid) - - # send an email to the people who missed out - cn = self.classname - title = self.get(nodeid, 'title') or '%s message copy'%cn - # figure author information - authid = messages.get(msgid, 'author') - authname = users.get(authid, 'realname') - if not authname: - authname = users.get(authid, 'username') - authaddr = users.get(authid, 'address') - if authaddr: - authaddr = " <%s>" % straddr( ('',authaddr) ) - else: - authaddr = '' - - # make the message body - m = [''] - - # put in roundup's signature - if self.db.config.EMAIL_SIGNATURE_POSITION == 'top': - m.append(self.email_signature(nodeid, msgid)) - - # add author information - if len(self.get(nodeid,'messages')) == 1: - m.append("New submission from %s%s:"%(authname, authaddr)) - else: - m.append("%s%s added the comment:"%(authname, authaddr)) - m.append('') - - # add the content - m.append(messages.get(msgid, 'content')) - - # add the change note - if note: - m.append(note) - - # put in roundup's signature - if self.db.config.EMAIL_SIGNATURE_POSITION == 'bottom': - m.append(self.email_signature(nodeid, msgid)) - - # encode the content as quoted-printable - content = cStringIO.StringIO('\n'.join(m)) - content_encoded = cStringIO.StringIO() - quopri.encode(content, content_encoded, 0) - content_encoded = content_encoded.getvalue() - - # get the files for this message - message_files = messages.get(msgid, 'files') - - # make sure the To line is always the same (for testing mostly) - sendto.sort() - - # create the message - message = cStringIO.StringIO() - writer = MimeWriter.MimeWriter(message) - writer.addheader('Subject', '[%s%s] %s'%(cn, nodeid, title)) - writer.addheader('To', ', '.join(sendto)) - writer.addheader('From', straddr( - (authname, self.db.config.TRACKER_EMAIL) ) ) - writer.addheader('Reply-To', straddr( - (self.db.config.TRACKER_NAME, - self.db.config.TRACKER_EMAIL) ) ) - writer.addheader('MIME-Version', '1.0') - if messageid: - writer.addheader('Message-Id', messageid) - if inreplyto: - writer.addheader('In-Reply-To', inreplyto) - - # add a uniquely Roundup header to help filtering - writer.addheader('X-Roundup-Name', self.db.config.TRACKER_NAME) - - # attach files - if message_files: - part = writer.startmultipartbody('mixed') - part = writer.nextpart() - part.addheader('Content-Transfer-Encoding', 'quoted-printable') - body = part.startbody('text/plain') - body.write(content_encoded) - for fileid in message_files: - name = files.get(fileid, 'name') - mime_type = files.get(fileid, 'type') - content = files.get(fileid, 'content') - part = writer.nextpart() - if mime_type == 'text/plain': - part.addheader('Content-Disposition', - 'attachment;\n filename="%s"'%name) - part.addheader('Content-Transfer-Encoding', '7bit') - body = part.startbody('text/plain') - body.write(content) - else: - # some other type, so encode it - if not mime_type: - # this should have been done when the file was saved - mime_type = mimetypes.guess_type(name)[0] - if mime_type is None: - mime_type = 'application/octet-stream' - part.addheader('Content-Disposition', - 'attachment;\n filename="%s"'%name) - part.addheader('Content-Transfer-Encoding', 'base64') - body = part.startbody(mime_type) - body.write(base64.encodestring(content)) - writer.lastpart() - else: - writer.addheader('Content-Transfer-Encoding', 'quoted-printable') - body = writer.startbody('text/plain') - body.write(content_encoded) - - # now try to send the message - if SENDMAILDEBUG: - open(SENDMAILDEBUG, 'w').write('FROM: %s\nTO: %s\n%s\n'%( - self.db.config.ADMIN_EMAIL, - ', '.join(sendto),message.getvalue())) - else: - try: - # send the message as admin so bounces are sent there - # instead of to roundup - smtp = smtplib.SMTP(self.db.config.MAILHOST) - smtp.sendmail(self.db.config.ADMIN_EMAIL, sendto, - message.getvalue()) - except socket.error, value: - raise MessageSendError, \ - "Couldn't send confirmation email: mailhost %s"%value - except smtplib.SMTPException, value: - raise MessageSendError, \ - "Couldn't send confirmation email: %s"%value - - def email_signature(self, nodeid, msgid): - ''' Add a signature to the e-mail with some useful information - ''' - # simplistic check to see if the url is valid, - # then append a trailing slash if it is missing - base = self.db.config.TRACKER_WEB - if not isinstance(base , type('')) or not base.startswith('http://'): - base = "Configuration Error: TRACKER_WEB isn't a " \ - "fully-qualified URL" - elif base[-1] != '/' : - base += '/' - web = base + self.classname + nodeid - - # ensure the email address is properly quoted - email = straddr((self.db.config.TRACKER_NAME, - self.db.config.TRACKER_EMAIL)) - - line = '_' * max(len(web), len(email)) - return '%s\n%s\n%s\n%s'%(line, email, web, line) - - - def generateCreateNote(self, nodeid): - """Generate a create note that lists initial property values - """ - cn = self.classname - cl = self.db.classes[cn] - props = cl.getprops(protected=0) - - # list the values - m = [] - l = props.items() - l.sort() - for propname, prop in l: - value = cl.get(nodeid, propname, None) - # skip boring entries - if not value: - continue - if isinstance(prop, hyperdb.Link): - link = self.db.classes[prop.classname] - if value: - key = link.labelprop(default_to_id=1) - if key: - value = link.get(value, key) - else: - value = '' - elif isinstance(prop, hyperdb.Multilink): - if value is None: value = [] - l = [] - link = self.db.classes[prop.classname] - key = link.labelprop(default_to_id=1) - if key: - value = [link.get(entry, key) for entry in value] - value.sort() - value = ', '.join(value) - m.append('%s: %s'%(propname, value)) - m.insert(0, '----------') - m.insert(0, '') - return '\n'.join(m) - - def generateChangeNote(self, nodeid, oldvalues): - """Generate a change note that lists property changes - """ - if __debug__ : - if not isinstance(oldvalues, type({})) : - raise TypeError("'oldvalues' must be dict-like, not %s."% - type(oldvalues)) - - cn = self.classname - cl = self.db.classes[cn] - changed = {} - props = cl.getprops(protected=0) - - # determine what changed - for key in oldvalues.keys(): - if key in ['files','messages']: - continue - if key in ('activity', 'creator', 'creation'): - continue - new_value = cl.get(nodeid, key) - # the old value might be non existent - try: - old_value = oldvalues[key] - if type(new_value) is type([]): - new_value.sort() - old_value.sort() - if new_value != old_value: - changed[key] = old_value - except: - changed[key] = new_value - - # list the changes - m = [] - l = changed.items() - l.sort() - for propname, oldvalue in l: - prop = props[propname] - value = cl.get(nodeid, propname, None) - if isinstance(prop, hyperdb.Link): - link = self.db.classes[prop.classname] - key = link.labelprop(default_to_id=1) - if key: - if value: - value = link.get(value, key) - else: - value = '' - if oldvalue: - oldvalue = link.get(oldvalue, key) - else: - oldvalue = '' - change = '%s -> %s'%(oldvalue, value) - elif isinstance(prop, hyperdb.Multilink): - change = '' - if value is None: value = [] - if oldvalue is None: oldvalue = [] - l = [] - link = self.db.classes[prop.classname] - key = link.labelprop(default_to_id=1) - # check for additions - for entry in value: - if entry in oldvalue: continue - if key: - l.append(link.get(entry, key)) - else: - l.append(entry) - if l: - l.sort() - change = '+%s'%(', '.join(l)) - l = [] - # check for removals - for entry in oldvalue: - if entry in value: continue - if key: - l.append(link.get(entry, key)) - else: - l.append(entry) - if l: - l.sort() - change += ' -%s'%(', '.join(l)) - else: - change = '%s -> %s'%(oldvalue, value) - m.append('%s: %s'%(propname, change)) - if m: - m.insert(0, '----------') - m.insert(0, '') - return '\n'.join(m) - -# vim: set filetype=python ts=4 sw=4 et si
--- a/roundup/scripts/roundup_server.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,303 +0,0 @@ -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -""" HTTP Server that serves roundup. - -$Id: roundup_server.py,v 1.12 2002-09-23 06:48:35 richard Exp $ -""" - -# python version check -from roundup import version_check - -import sys, os, urllib, StringIO, traceback, cgi, binascii, getopt, imp -import BaseHTTPServer - -# Roundup modules of use here -from roundup.cgi import cgitb, client -import roundup.instance -from roundup.i18n import _ - -# -## Configuration -# - -# This indicates where the Roundup trackers live. They're given as NAME -> -# TRACKER_HOME, where the NAME part is used in the URL to select the -# appropriate reacker. -# Make sure the NAME part doesn't include any url-unsafe characters like -# spaces, as these confuse the cookie handling in browsers like IE. -TRACKER_HOMES = { - 'bar': '/tmp/bar', -} - -ROUNDUP_USER = None - - -# Where to log debugging information to. Use an instance of DevNull if you -# don't want to log anywhere. -# TODO: actually use this stuff -#class DevNull: -# def write(self, info): -# pass -#LOG = open('/var/log/roundup.cgi.log', 'a') -#LOG = DevNull() - -# -## end configuration -# - -class RoundupRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler): - TRACKER_HOMES = TRACKER_HOMES - ROUNDUP_USER = ROUNDUP_USER - - def run_cgi(self): - """ Execute the CGI command. Wrap an innner call in an error - handler so all errors can be caught. - """ - save_stdin = sys.stdin - sys.stdin = self.rfile - try: - self.inner_run_cgi() - except client.NotFound: - self.send_error(404, self.path) - except client.Unauthorised: - self.send_error(403, self.path) - except: - # it'd be nice to be able to detect if these are going to have - # any effect... - self.send_response(400) - self.send_header('Content-Type', 'text/html') - self.end_headers() - try: - reload(cgitb) - self.wfile.write(cgitb.breaker()) - self.wfile.write(cgitb.html()) - except: - self.wfile.write("<pre>") - s = StringIO.StringIO() - traceback.print_exc(None, s) - self.wfile.write(cgi.escape(s.getvalue())) - self.wfile.write("</pre>\n") - sys.stdin = save_stdin - - do_GET = do_POST = do_HEAD = send_head = run_cgi - - def index(self): - ''' Print up an index of the available instances - ''' - self.send_response(200) - self.send_header('Content-Type', 'text/html') - self.end_headers() - w = self.wfile.write - w(_('<html><head><title>Roundup instances index</title></head>\n')) - w(_('<body><h1>Roundup instances index</h1><ol>\n')) - for instance in self.TRACKER_HOMES.keys(): - w(_('<li><a href="%(instance_url)s/index">%(instance_name)s</a>\n')%{ - 'instance_url': urllib.quote(instance), - 'instance_name': cgi.escape(instance)}) - w(_('</ol></body></html>')) - - def inner_run_cgi(self): - ''' This is the inner part of the CGI handling - ''' - - rest = self.path - i = rest.rfind('?') - if i >= 0: - rest, query = rest[:i], rest[i+1:] - else: - query = '' - - # figure the instance - if rest == '/': - return self.index() - l_path = rest.split('/') - instance_name = urllib.unquote(l_path[1]) - if self.TRACKER_HOMES.has_key(instance_name): - instance_home = self.TRACKER_HOMES[instance_name] - instance = roundup.instance.open(instance_home) - else: - raise client.NotFound - - # figure out what the rest of the path is - if len(l_path) > 2: - rest = '/'.join(l_path[2:]) - else: - rest = '/' - - # Set up the CGI environment - env = {} - env['TRACKER_NAME'] = instance_name - env['REQUEST_METHOD'] = self.command - env['PATH_INFO'] = urllib.unquote(rest) - if query: - env['QUERY_STRING'] = query - host = self.address_string() - if self.headers.typeheader is None: - env['CONTENT_TYPE'] = self.headers.type - else: - env['CONTENT_TYPE'] = self.headers.typeheader - length = self.headers.getheader('content-length') - if length: - env['CONTENT_LENGTH'] = length - co = filter(None, self.headers.getheaders('cookie')) - if co: - env['HTTP_COOKIE'] = ', '.join(co) - env['SCRIPT_NAME'] = '' - env['SERVER_NAME'] = self.server.server_name - env['SERVER_PORT'] = str(self.server.server_port) - env['HTTP_HOST'] = self.headers['host'] - - decoded_query = query.replace('+', ' ') - - # do the roundup thang - c = instance.Client(instance, self, env) - c.main() - -def usage(message=''): - if message: - message = _('Error: %(error)s\n\n')%{'error': message} - print _('''%(message)sUsage: -roundup-server [-n hostname] [-p port] [-l file] [-d file] [name=instance home]* - - -n: sets the host name - -p: sets the port to listen on - -l: sets a filename to log to (instead of stdout) - -d: daemonize, and write the server's PID to the nominated file - - name=instance home - Sets the instance home(s) to use. The name is how the instance is - identified in the URL (it's the first part of the URL path). The - instance home is the directory that was identified when you did - "roundup-admin init". You may specify any number of these name=home - pairs on the command-line. For convenience, you may edit the - TRACKER_HOMES variable in the roundup-server file instead. - Make sure the name part doesn't include any url-unsafe characters like - spaces, as these confuse the cookie handling in browsers like IE. -''')%locals() - sys.exit(0) - -def daemonize(pidfile): - ''' Turn this process into a daemon. - - make sure the sys.std(in|out|err) are completely cut off - - make our parent PID 1 - - Write our new PID to the pidfile. - - From A.M. Kuuchling (possibly originally Greg Ward) with - modification from Oren Tirosh, and finally a small mod from me. - ''' - # Fork once - if os.fork() != 0: - os._exit(0) - - # Create new session - os.setsid() - - # Second fork to force PPID=1 - pid = os.fork() - if pid: - pidfile = open(pidfile, 'w') - pidfile.write(str(pid)) - pidfile.close() - os._exit(0) - - os.chdir("/") - os.umask(0) - - # close off sys.std(in|out|err), redirect to devnull so the file - # descriptors can't be used again - devnull = os.open('/dev/null', 0) - os.dup2(devnull, 0) - os.dup2(devnull, 1) - os.dup2(devnull, 2) - -def run(): - hostname = '' - port = 8080 - pidfile = None - logfile = None - try: - # handle the command-line args - try: - optlist, args = getopt.getopt(sys.argv[1:], 'n:p:u:d:l:') - except getopt.GetoptError, e: - usage(str(e)) - - user = ROUNDUP_USER - for (opt, arg) in optlist: - if opt == '-n': hostname = arg - elif opt == '-p': port = int(arg) - elif opt == '-u': user = arg - elif opt == '-d': pidfile = arg - elif opt == '-l': logfile = arg - elif opt == '-h': usage() - - if hasattr(os, 'getuid'): - # if root, setuid to the running user - if not os.getuid() and user is not None: - try: - import pwd - except ImportError: - raise ValueError, _("Can't change users - no pwd module") - try: - uid = pwd.getpwnam(user)[2] - except KeyError: - raise ValueError, _("User %(user)s doesn't exist")%locals() - os.setuid(uid) - elif os.getuid() and user is not None: - print _('WARNING: ignoring "-u" argument, not root') - - # People can remove this check if they're really determined - if not os.getuid() and user is None: - raise ValueError, _("Can't run as root!") - - # handle instance specs - if args: - d = {} - for arg in args: - try: - name, home = arg.split('=') - except ValueError: - raise ValueError, _("Instances must be name=home") - d[name] = home - RoundupRequestHandler.TRACKER_HOMES = d - except SystemExit: - raise - except: - exc_type, exc_value = sys.exc_info()[:2] - usage('%s: %s'%(exc_type, exc_value)) - - # we don't want the cgi module interpreting the command-line args ;) - sys.argv = sys.argv[:1] - address = (hostname, port) - - # fork? - if pidfile: - daemonize(pidfile) - - # redirect stdout/stderr to our logfile - if logfile: - sys.stdout = sys.stderr = open(logfile, 'a') - - httpd = BaseHTTPServer.HTTPServer(address, RoundupRequestHandler) - print _('Roundup server started on %(address)s')%locals() - httpd.serve_forever() - -if __name__ == '__main__': - run() - -# vim: set filetype=python ts=4 sw=4 et si
--- a/roundup/templates/classic/config.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,92 +0,0 @@ -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: config.py,v 1.3 2002-09-11 01:18:42 richard Exp $ - -import os - -# roundup home is this package's directory -TRACKER_HOME=os.path.split(__file__)[0] - -# The SMTP mail host that roundup will use to send mail -MAILHOST = 'localhost' - -# The domain name used for email addresses. -MAIL_DOMAIN = 'your.tracker.email.domain.example' - -# This is the directory that the database is going to be stored in -DATABASE = os.path.join(TRACKER_HOME, 'db') - -# This is the directory that the HTML templates reside in -TEMPLATES = os.path.join(TRACKER_HOME, 'html') - -# A descriptive name for your roundup instance -TRACKER_NAME = 'Roundup issue tracker' - -# The email address that mail to roundup should go to -TRACKER_EMAIL = 'issue_tracker@%s'%MAIL_DOMAIN - -# The web address that the instance is viewable at -TRACKER_WEB = 'http://your.tracker.url.example/' - -# The email address that roundup will complain to if it runs into trouble -ADMIN_EMAIL = 'roundup-admin@%s'%MAIL_DOMAIN - -# Where to place the web filtering HTML on the index page -FILTER_POSITION = 'bottom' # one of 'top', 'bottom', 'top and bottom' - -# -# SECURITY DEFINITIONS -# -# define the Roles that a user gets when they register with the tracker -# these are a comma-separated string of role names (e.g. 'Admin,User') -NEW_WEB_USER_ROLES = 'User' -NEW_EMAIL_USER_ROLES = 'User' - -# Send nosy messages to the author of the message -MESSAGES_TO_AUTHOR = 'no' # either 'yes' or 'no' - -# Does the author of a message get placed on the nosy list automatically? -# If 'new' is used, then the author will only be added when a message -# creates a new issue. If 'yes', then the author will be added on followups -# too. If 'no', they're never added to the nosy. -ADD_AUTHOR_TO_NOSY = 'new' # one of 'yes', 'no', 'new' - -# Do the recipients (To:, Cc:) of a message get placed on the nosy list? -# If 'new' is used, then the recipients will only be added when a message -# creates a new issue. If 'yes', then the recipients will be added on followups -# too. If 'no', they're never added to the nosy. -ADD_RECIPIENTS_TO_NOSY = 'new' # either 'yes', 'no', 'new' - -# Where to place the email signature -EMAIL_SIGNATURE_POSITION = 'bottom' # one of 'top', 'bottom', 'none' - -# Keep email citations when accepting messages. Setting this to "no" strips -# out "quoted" text from the message. Signatures are also stripped. -EMAIL_KEEP_QUOTED_TEXT = 'yes' # either 'yes' or 'no' - -# Preserve the email body as is - that is, keep the citations _and_ -# signatures. -EMAIL_LEAVE_BODY_UNCHANGED = 'no' # either 'yes' or 'no' - -# Default class to use in the mailgw if one isn't supplied in email -# subjects. To disable, comment out the variable below or leave it blank. -# Examples: -MAIL_DEFAULT_CLASS = 'issue' # use "issue" class by default -#MAIL_DEFAULT_CLASS = '' # disable (or just comment the var out) - -# vim: set filetype=python ts=4 sw=4 et si
--- a/roundup/templates/classic/dbinit.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,193 +0,0 @@ -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: dbinit.py,v 1.29 2002-09-13 03:31:18 richard Exp $ - -import os - -import config -from select_db import Database, Class, FileClass, IssueClass - -def open(name=None): - ''' as from the roundupdb method openDB - ''' - from roundup.hyperdb import String, Password, Date, Link, Multilink - - # open the database - db = Database(config, name) - - # - # Now initialise the schema. Must do this each time the database is - # opened. - # - - # Class automatically gets these properties: - # creation = Date() - # activity = Date() - # creator = Link('user') - pri = Class(db, "priority", - name=String(), order=String()) - pri.setkey("name") - - stat = Class(db, "status", - name=String(), order=String()) - stat.setkey("name") - - keyword = Class(db, "keyword", - name=String()) - keyword.setkey("name") - - query = Class(db, "query", - klass=String(), name=String(), - url=String()) - query.setkey("name") - - # Note: roles is a comma-separated string of Role names - user = Class(db, "user", - username=String(), password=Password(), - address=String(), realname=String(), - phone=String(), organisation=String(), - alternate_addresses=String(), - queries=Multilink('query'), roles=String()) - user.setkey("username") - - # FileClass automatically gets these properties: - # content = String() [saved to disk in <tracker home>/db/files/] - # (it also gets the Class properties creation, activity and creator) - msg = FileClass(db, "msg", - author=Link("user", do_journal='no'), - recipients=Multilink("user", do_journal='no'), - date=Date(), summary=String(), - files=Multilink("file"), - messageid=String(), inreplyto=String()) - - file = FileClass(db, "file", - name=String(), type=String()) - - # IssueClass automatically gets these properties: - # title = String() - # messages = Multilink("msg") - # files = Multilink("file") - # nosy = Multilink("user") - # superseder = Multilink("issue") - # (it also gets the Class properties creation, activity and creator) - issue = IssueClass(db, "issue", - assignedto=Link("user"), topic=Multilink("keyword"), - priority=Link("priority"), status=Link("status")) - - # - # SECURITY SETTINGS - # - # new permissions for this schema - for cl in 'issue', 'file', 'msg', 'user', 'query', 'keyword': - db.security.addPermission(name="Edit", klass=cl, - description="User is allowed to edit "+cl) - db.security.addPermission(name="View", klass=cl, - description="User is allowed to access "+cl) - - # Assign the access and edit permissions for issue, file and message - # to regular users now - for cl in 'issue', 'file', 'msg', 'query', 'keyword': - p = db.security.getPermission('View', cl) - db.security.addPermissionToRole('User', p) - p = db.security.getPermission('Edit', cl) - db.security.addPermissionToRole('User', p) - # and give the regular users access to the web and email interface - p = db.security.getPermission('Web Access') - db.security.addPermissionToRole('User', p) - p = db.security.getPermission('Email Access') - db.security.addPermissionToRole('User', p) - - # May users view other user information? Comment these lines out - # if you don't want them to - p = db.security.getPermission('View', 'user') - db.security.addPermissionToRole('User', p) - - # Assign the appropriate permissions to the anonymous user's Anonymous - # Role. Choices here are: - # - Allow anonymous users to register through the web - p = db.security.getPermission('Web Registration') - db.security.addPermissionToRole('Anonymous', p) - # - Allow anonymous (new) users to register through the email gateway - p = db.security.getPermission('Email Registration') - db.security.addPermissionToRole('Anonymous', p) - # - Allow anonymous users access to the "issue" class of data - # Note: this also grants access to related information like files, - # messages, statuses etc that are linked to issues - p = db.security.getPermission('View', 'issue') - db.security.addPermissionToRole('Anonymous', p) - # - Allow anonymous users access to edit the "issue" class of data - # Note: this also grants access to create related information like - # files and messages etc that are linked to issues - #p = db.security.getPermission('Edit', 'issue') - #db.security.addPermissionToRole('Anonymous', p) - - # oh, g'wan, let anonymous access the web interface too - p = db.security.getPermission('Web Access') - db.security.addPermissionToRole('Anonymous', p) - - import detectors - detectors.init(db) - - # schema is set up - run any post-initialisation - db.post_init() - return db - -def init(adminpw): - ''' as from the roundupdb method initDB - - Open the new database, and add new nodes - used for initialisation. You - can edit this before running the "roundup-admin initialise" command to - change the initial database entries. - ''' - dbdir = os.path.join(config.DATABASE, 'files') - if not os.path.isdir(dbdir): - os.makedirs(dbdir) - - db = open("admin") - db.clear() - - # - # INITIAL PRIORITY AND STATUS VALUES - # - pri = db.getclass('priority') - pri.create(name="critical", order="1") - pri.create(name="urgent", order="2") - pri.create(name="bug", order="3") - pri.create(name="feature", order="4") - pri.create(name="wish", order="5") - - stat = db.getclass('status') - stat.create(name="unread", order="1") - stat.create(name="deferred", order="2") - stat.create(name="chatting", order="3") - stat.create(name="need-eg", order="4") - stat.create(name="in-progress", order="5") - stat.create(name="testing", order="6") - stat.create(name="done-cbb", order="7") - stat.create(name="resolved", order="8") - - # create the two default users - user = db.getclass('user') - user.create(username="admin", password=adminpw, - address=config.ADMIN_EMAIL, roles='Admin') - user.create(username="anonymous", roles='Anonymous') - - db.commit() - -# vim: set filetype=python ts=4 sw=4 et si -
--- a/roundup/templates/classic/detectors/statusauditor.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,66 +0,0 @@ -# Copyright (c) 2002 ekit.com Inc (http://www.ekit-inc.com/) -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -# SOFTWARE. -# -#$Id: statusauditor.py,v 1.2 2002-09-10 01:07:06 richard Exp $ - -def chatty(db, cl, nodeid, newvalues): - ''' If the issue is currently 'unread' or 'resolved', then set - it to 'chatting' - ''' - # don't fire if there's no new message (ie. chat) - if not newvalues.has_key('messages'): - return - if newvalues['messages'] == cl.get(nodeid, 'messages', cache=0): - return - - # determine the id of 'unread', 'resolved' and 'chatting' - unread_id = db.status.lookup('unread') - resolved_id = db.status.lookup('resolved') - chatting_id = db.status.lookup('chatting') - - # get the current value - current_status = cl.get(nodeid, 'status') - - # see if there's an explicit change in this transaction - if newvalues.has_key('status') and newvalues['status'] != current_status: - # yep, skip - return - - # ok, there's no explicit change, so do it manually - if current_status in (unread_id, resolved_id): - newvalues['status'] = chatting_id - - -def presetunread(db, cl, nodeid, newvalues): - ''' Make sure the status is set on new issues - ''' - if newvalues.has_key('status'): - return - - # ok, do it - newvalues['status'] = db.status.lookup('unread') - - -def init(db): - # fire before changes are made - db.issue.audit('set', chatty) - db.issue.audit('create', presetunread) - -# vim: set filetype=python ts=4 sw=4 et si
--- a/roundup/templates/classic/html/issue.index Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,110 +0,0 @@ -<!-- dollarId: issue.index,v 1.2 2001/07/29 04:07:37 richard Exp dollar--> -<tal:block metal:use-macro="templates/page/macros/icing"> -<title metal:fill-slot="head_title">List of issues</title> -<td class="page-header-top" metal:fill-slot="body_title"> - <h2>List of issues</h2> -</td> -<td class="content" metal:fill-slot="content"> - -<tal:block tal:condition="not:context/is_view_ok"> -You are not allowed to view this page. -</tal:block> - -<tal:block tal:define="batch request/batch" tal:condition="context/is_view_ok"> - <table class="list"> - <tr> - <th tal:condition="request/show/priority">Priority</th> - <th tal:condition="request/show/id">ID</th> - <th tal:condition="request/show/activity">Activity</th> - <th tal:condition="request/show/topic">Topic</th> - <th tal:condition="request/show/title">Title</th> - <th tal:condition="request/show/status">Status</th> - <th tal:condition="request/show/creator">Created By</th> - <th tal:condition="request/show/assignedto">Assigned To</th> - </tr> - <tal:block tal:repeat="i batch"> - <tr tal:condition="python:request.group[1] and - batch.propchanged(request.group[1])"> - <th tal:attributes="colspan python:len(request.columns)" - tal:content="python:i[request.group[1]]" class="group"> - </th> - </tr> - <tr tal:attributes="class python:['normal', 'alt'][repeat['i'].even()]"> - <td tal:condition="request/show/priority" tal:content="i/priority"></td> - <td tal:condition="request/show/id" tal:content="i/id"></td> - <td nowrap tal:condition="request/show/activity" - tal:content="i/activity/reldate"></td> - <td tal:condition="request/show/topic" tal:content="i/topic"></td> - <td tal:condition="request/show/title"> - <a tal:attributes="href string:issue${i/id}" - tal:content="python:str(i.title) or '[no title]'">title</a> - </td> - <td tal:condition="request/show/status" tal:content="i/status"></td> - <td tal:condition="request/show/creator" tal:content="i/creator"></td> - <td tal:condition="request/show/assignedto" tal:content="i/assignedto"></td> - </tr> - </tal:block> - <tr> - <td style="padding: 0" tal:attributes="colspan python:len(request.columns)"> - <table class="list"> - <tr><th style="text-align: left; border: 0"> - <a tal:define="prev batch/previous" tal:condition="prev" - tal:attributes="href python:request.indexargs_href(request.classname, - {':startwith':prev.first, ':pagesize':prev.size})"><< previous</a> - - </th> - <th style="text-align: right; border: 0"> - <a tal:define="next batch/next" tal:condition="next" - tal:attributes="href python:request.indexargs_href(request.classname, - {':startwith':next.first, ':pagesize':next.size})">next >></a> - - </th></tr> - </table> - </td> - </tr> -</table> - -<form method="GET" tal:attributes="action request/classname"> - <tal:block tal:replace="structure python:request.indexargs_form(sort=0, group=0)" /> - <table class="form"> - <tr tal:condition="batch"> - <th>Sort on:</th> - <td> - <select name=":sort"> - <option value="">- nothing -</option> - <option tal:repeat="col context/properties" - tal:attributes="value col/_name; - selected python:col._name == request.sort[1]" - tal:content="col/_name">column</option> - </select> - </td> - <th>Descending:</th> - <td><input type="checkbox" name=":sortdir" - tal:attributes="checked python:request.sort[0] == '-'"> - </td> - </tr> - <tr> - <th>Group on:</th> - <td> - <select name=":group"> - <option value="">- nothing -</option> - <option tal:repeat="col context/properties" - tal:attributes="value col/_name; - selected python:col._name == request.group[1]" - tal:content="col/_name">column</option> - </select> - </td> - <th>Descending:</th> - <td><input type="checkbox" name=":groupdir" - tal:attributes="checked python:request.group[0] == '-'"> - </td> - </tr> - <tr><td colspan="4"><input type="submit" value="Redisplay"></td></tr> - </table> -</form> - -</tal:block> - -</td> -</tal:block> -
--- a/roundup/templates/classic/html/issue.item Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,148 +0,0 @@ -<!-- dollarId: issue.item,v 1.4 2001/08/03 01:19:43 richard Exp dollar--> -<tal:block metal:use-macro="templates/page/macros/icing"> -<title metal:fill-slot="head_title">Issue editing</title> -<td class="page-header-top" metal:fill-slot="body_title"><h2>Issue Editing</h2> -</td> - -<td class="content" metal:fill-slot="content"> - -<span tal:condition="python:not (context.is_view_ok() or context.is_edit_ok())"> -You are not allowed to view this page. -</span> - -<form method="POST" onSubmit="return submit_once()" - enctype="multipart/form-data" tal:condition="context/is_edit_ok"> - -<input type="hidden" name=":template" value="item"> -<input type="hidden" name=":required" value="title,priority"> - -<table class="form"> -<tr> - <th nowrap>Title</th> - <td colspan=3 tal:content="structure python:context.title.field(size=60)">title</td> -</tr> - -<tr> - <th nowrap>Priority</th> - <td tal:content="structure context/priority/menu">priority</td> - <th nowrap>Status</th> - <td tal:content="structure context/status/menu">status</td> -</tr> - -<tr> - <th nowrap>Superseder</th> - <td> - <span tal:replace="structure python:context.superseder.field(showid=1, size=20)" /> - <span tal:replace="structure python:db.issue.classhelp('id,title')" /> - <span tal:condition="context/superseder" tal:repeat="sup context/superseder"> - <br>View: <a tal:attributes="href string:issue${sup/id}" - tal:content="sup/id"></a> - </span> - </td> - <th nowrap>Nosy List</th> - <td> - <span tal:replace="structure context/nosy/field" /> - <span tal:replace="structure python:db.user.classhelp('username,realname,address,phone')" /> - </td> -</tr> - -<tr> - <th nowrap>Assigned To</th> - <td tal:content="structure context/assignedto/menu">assignedto menu</td> - <th nowrap>Topics</th> - <td> - <span tal:replace="structure context/topic/field" /> - <span tal:replace="structure db/keyword/classhelp" /> - </td> -</tr> - -<tr> - <th nowrap>Change Note</th> - <td colspan=3> - <textarea name=":note" wrap="hard" rows="5" cols="60"></textarea> - </td> -</tr> - -<tr> - <th nowrap>File</th> - <td colspan=3><input type="file" name=":file" size="40"></td> -</tr> - -<tr> - <td> </td> - <td colspan=3 tal:content="structure context/submit"> - submit button will go here - </td> -</tr> -</table> - -</form> - -<table class="form" tal:condition="context/is_only_view_ok"> -<tr> - <th nowrap>Title</th><td colspan=3 tal:content="context/title">title</td> -</tr> - -<tr> - <th nowrap>Priority</th><td tal:content="context/priority">priority</td> - <th nowrap>Status</th><td tal:content="context/status">status</td> -</tr> - -<tr> - <th nowrap>Superseder</th> - <td> - <span tal:condition="context/superseder" tal:repeat="sup context/superseder"> - <br>View: <a tal:attributes="href string:issue${sup/id}" - tal:content="sup/id"></a> - </span> - </td> - <th nowrap>Nosy List</th><td><span tal:replace="context/nosy" /></td> -</tr> - -<tr> - <th nowrap>Assigned To</th><td tal:content="context/assignedto"></td> - <th nowrap>Topics</th><td tal:content="structure context/topic"></td> -</tr> -</table> - -<tal:block tal:condition="python:context.id and context.is_view_ok()"> - - <p tal:content="structure string:Created on - <b>${context/creation}</b> by <b>${context/creator}</b>, last - changed <b>${context/activity}</b>.">activity info - </p> - - <table class="messages" tal:condition="context/messages"> - <tr><th colspan=4 class="header">Messages</th></tr> - <tr><th>Message</th><th>Author</th><th>Date</th><th>Summary</th></tr> - <tr tal:repeat="msg context/messages/reverse"> - <td><a tal:attributes="href string:msg${msg/id}" - tal:content="string:msg${msg/id}"></a></td> - <td tal:content="msg/author">author</td> - <td tal:content="msg/date">date</td> - <td tal:content="msg/summary">summary</td> - </tr> - </table> - - <table class="files" tal:condition="context/files"> - <tr><th colspan="2" class="header">Files</th></tr> - <tr><th>File name</th><th>Uploaded</th></tr> - <tr tal:repeat="file context/files"> - <td> - <a tal:attributes="href string:file${file/id}/${file/name}" - tal:content="file/name">dld link</a> - </td> - <td> - <span tal:content="file/creator">creator's name</span>, - <span tal:content="file/creation">creation date</span> - </td> - </tr> - </table> - - <tal:block tal:replace="structure context/history" /> - -</tal:block> - -</td> - -</tal:block>
--- a/roundup/templates/classic/html/issue.search Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,184 +0,0 @@ -<tal:block metal:use-macro="templates/page/macros/icing"> -<title metal:fill-slot="head_title">Issue searching</title> -<td class="page-header-top" metal:fill-slot="body_title"> - <h2>Issue searching</h2> -</td> -<td class="content" metal:fill-slot="content"> - -<form method="GET" tal:attributes="action request/classname"> -<input type="hidden" name=":action" value="search"> - -<table class="form" tal:define=" - cols python:'id activity priority title status assignedto'.split(); - defsort python:['activity']; - defgroup python:['priority']; - defdisp python:'id activity title status assignedto'.split()"> - -<tr> - <th class="header"> </th> - <th class="header">Filter on</th> - <th class="header">Display</th> - <th class="header">Sort on</th> - <th class="header">Group on</th> -</tr> - -<tr> - <th>All text*:</th> - <td><input name=":search_text" - tal:attributes="value request/form/:search_text/value | nothing"> - </td> - <td> </td> - <td> </td> - <td> </td> -</tr> - -<tr> - <th>Title:</th> - <td><input name="title"></td> - <td><input type="checkbox" name=":columns" value="title" checked></td> - <td><input type="radio" name=":sort" value="title"></td> - <td> </td> -</tr> - -<tr> - <th>Topic:</th> - <td> - <select name="topic"> - <option value="">don't care</option> - <option value="">------------</option> - <option tal:repeat="s db/keyword/list" tal:attributes="value s/name" - tal:content="s/name">topic to filter on</option> - </select> - </td> - <td><input type="checkbox" name=":columns" value="topic" checked></td> - <td><input type="radio" name=":sort" value="topic"></td> - <td><input type="radio" name=":group" value="topic"></td> -</tr> - -<tr> - <th>Created:</th> - <td><input name="activity"></td> - <td><input type="checkbox" name=":columns" value="created"></td> - <td><input type="radio" name=":sort" value="created"></td> - <td><input type="radio" name=":group" value="created"></td> -</tr> - -<tr> - <th>Creator:</th> - <td> - <select name="creator"> - <option value="">don't care</option> - <option tal:attributes="value request/user/id">created by me</option> - <option value="-1">------------</option> - <option tal:repeat="s db/user/list" tal:attributes="value s/id" - tal:content="s/username">user to filter on</option> - </select> - </td> - <td><input type="checkbox" name=":columns" value="creator" checked></td> - <td><input type="radio" name=":sort" value="creator"></td> - <td><input type="radio" name=":group" value="creator"></td> -</tr> - -<tr> - <th>Activity:</th> - <td><input name="activity"></td> - <td><input type="checkbox" name=":columns" value="activity" checked></td> - <td><input type="radio" name=":sort" value="activity"></td> - <td> </td> -</tr> - -<tr> - <th>Priority:</th> - <td> - <select name="priority"> - <option value="">don't care</option> - <option value="-1">not selected</option> - <option value="">------------</option> - <option tal:repeat="s db/priority/list" tal:attributes="value s/id" - tal:content="s/name">priority to filter on</option> - </select> - </td> - <td><input type="checkbox" name=":columns" value="priority"></td> - <td><input type="radio" name=":sort" value="priority"></td> - <td><input type="radio" name=":group" value="priority"></td> -</tr> - -<tr> - <th>Status:</th> - <td> - <select name="status"> - <option value="">don't care</option> - <option value="-1,1,2,3,4,5,6,7">not resolved</option> - <option value="-1">not selected</option> - <option value="">------------</option> - <option tal:repeat="s db/status/list" tal:attributes="value s/id" - tal:content="s/name">status to filter on</option> - </select> - </td> - <td><input type="checkbox" name=":columns" value="status" checked></td> - <td><input type="radio" name=":sort" value="status"></td> - <td><input type="radio" name=":group" value="status"></td> -</tr> - -<tr> - <th>Assigned To:</th> - <td> - <select name="assignedto"> - <option value="">don't care</option> - <option tal:attributes="value request/user/id">assigned to me</option> - <option value="-1">unassigned</option> - <option value="">------------</option> - <option tal:repeat="s db/user/list" tal:attributes="value s/id" - tal:content="s/username">user to filter on</option> - </select> - </td> - <td><input type="checkbox" name=":columns" value="assignedto" checked></td> - <td><input type="radio" name=":sort" value="assignedto"></td> - <td><input type="radio" name=":group" value="assignedto"></td> -</tr> - -<tr> -<th>Pagesize:</th> -<td><input type="text" name=":pagesize" size="3" value="50"></td> -</tr> - -<tr> -<th>Start With:</th> -<td><input type="text" name=":startwith" size="3" value="0"></td> -</tr> - -<tr> -<th>Sort Descending:</th> -<td><input type="checkbox" name=":sortdir" checked> -</td> - -<tr> -<th>Group Descending:</th> -<td><input type="checkbox" name=":groupdir"> -</td> -</tr> - -<tr> -<th>Query name**:</th> -<td><input name=":queryname" - tal:attributes="value request/form/:queryname/value | nothing"> -</td> -</tr> - -<tr><td> </td> -<td><input type="submit" value="Search"></td> -</tr> - -<tr><td> </td> - <td colspan="4" class="help"> - *: The "all text" field will look in message bodies and issue titles<br> - **: If you supply a name, the query will be saved off and available as a - link in the sidebar - </td> -</tr> -</table> - -</form> -</td> - -</tal:block>
--- a/roundup/templates/classic/html/query.item Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,16 +0,0 @@ -<tal:block metal:use-macro="templates/page/macros/icing"> -<title metal:fill-slot="head_title">Query editing</title> -<td class="page-header-top" metal:fill-slot="body_title"> - <h2>Query editing</h2> -</td> -<td class="content" metal:fill-slot="content"> -<span tal:condition="not:context/is_edit_ok"> -You are not allowed to view this page. -</span> - -<span tal:condition="context/is_edit_ok" - tal:content="structure context/renderQueryForm" /> - -</td> - -</tal:block>
--- a/roundup/templates/classic/html/style.css Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,305 +0,0 @@ -/* main page styles */ -body.body { - font-family: sans-serif, Arial, Helvetica; - color: #333333; -} -a[href]:hover { color:blue; text-decoration: underline; } -a[href]:link { color:blue; text-decoration: none; } -a[href] { color:blue; text-decoration: none; } - -table.body { - border: 0; - padding: 0; - border-spacing: 0px; - border-collapse: separate; -} - -td.page-header-left { - padding: 5px; - border-bottom: 1px solid #444444; -} - -td.page-header-top { - border-bottom: 1px solid #444444; - padding: 5px; -} - -td.sidebar { - padding: 1 0 0 1; -} - -td.sidebar p.classblock { - padding: 0 5 0 5; - margin: 1 1 1 1; - border: 1px solid #444444; - background-color: #eeeeee; -} - -td.sidebar p.userblock { - padding: 0 5 0 5; - margin: 1 1 1 1; - border: 1px solid #444444; - background-color: #eeeeff; -} - -td.content { - padding: 1 5 1 5; - vertical-align: top; -} - -p.ok-message { - background-color: #22bb22; - padding: 5 5 5 5; - color: white; - font-weight: bold; -} -p.error-message { - background-color: #bb2222; - padding: 5 5 5 5; - color: white; - font-weight: bold; -} - - -/* style for forms */ -table.form { - padding: 2; - border-spacing: 0px; - border-collapse: separate; -} - -table.form th { - font-weight: bold; - color: #333388; - text-align: right; - vertical-align: top; -} - -table.form th.header { - font-weight: bold; - color: #333388; - background-color: #eeeeff; - text-align: left; -} - -table.form td.optional { - font-weight: bold; - font-style: italic; - color: #333333; - empty-cells: show; -} - -table.form td { - color: #333333; - empty-cells: show; -} - -table.form td.html { - color: #777777; - empty-cells: show; -} - -/* style for lists */ -table.list { - border-spacing: 0px; - border-collapse: separate; - width: 100%; -} - -table.list th { - padding: 0 4 0 4; - color: #404070; - background-color: #eeeeff; - border-right: 1px solid #404070; - border-top: 1px solid #404070; - border-bottom: 1px solid #404070; - vertical-align: top; -} -table.list th a[href]:hover { color: #404070 } -table.list th a[href]:link { color: #404070 } -table.list th a[href] { color: #404070 } -table.list th.group { - background-color: #f4f4ff; - text-align: center; -} - -table.list td { - padding: 0 4 0 4; - border: 0 2 0 2; - border-right: 1px solid #404070; - color: #404070; - background-color: white; - vertical-align: top; -} - -table.list tr.normal td { - empty-cells: show; -} - -table.list tr.alt td { - background-color: #efefef; - empty-cells: show; -} - -table.list td:first-child { - border-left: 1px solid #404070; - border-right: 1px solid #404070; - empty-cells: show; -} - -table.list th:first-child { - border-left: 1px solid #404070; - border-right: 1px solid #404070; - empty-cells: show; -} - - -/* style for message displays */ -table.messages { - border-spacing: 0px; - border-collapse: separate; - width: 100%; -} - -table.messages th.header{ - padding-top: 10px; - border-bottom: 1px solid gray; - font-weight: bold; - background-color: white; - color: #707040; -} - -table.messages th { - font-weight: bold; - color: black; - text-align: left; -} - -table.messages td { - font-family: monospace; - background-color: #efefef; - border-top: 1px solid #afafaf; - border-bottom: 1px solid #afafaf; - color: black; - empty-cells: show; -} - -/* style for file displays */ -table.files { - border-spacing: 0px; - border-collapse: separate; - width: 100%; -} - -table.files th.header{ - padding-top: 10px; - border-bottom: 1px solid gray; - font-weight: bold; - background-color: white; - color: #707040; -} - -table.files th { - border-bottom: 1px solid #afafaf; - font-weight: bold; - text-align: left; -} - -table.files td { - font-family: monospace; - empty-cells: show; -} - -/* style for history displays */ -table.history { - border-spacing: 0px; - border-collapse: separate; - width: 100%; -} - -table.history th.header{ - padding-top: 10px; - border-bottom: 1px solid gray; - font-weight: bold; - background-color: white; - color: #707040; - font-size: 100%; -} - -table.history th { - border-bottom: 1px solid #afafaf; - font-weight: bold; - text-align: left; - font-size: 90%; -} - -table.history td { - font-size: 90%; - vertical-align: top; - empty-cells: show; -} - - -/* style for class list */ -table.classlist { - border-spacing: 0px; - border-collapse: separate; - width: 100%; -} - -table.classlist th.header{ - padding-top: 10px; - border-bottom: 1px solid gray; - font-weight: bold; - background-color: white; - color: #707040; -} - -table.classlist th { - font-weight: bold; - text-align: left; -} - - -/* style for class help display */ -table.classhelp { - border-spacing: 0px; - border-collapse: separate; - width: 100%; -} - -table.classhelp th { - font-weight: bold; - text-align: left; - color: #707040; -} - -table.classhelp td { - padding: 2 2 2 2; - border: 1px solid black; - text-align: left; - vertical-align: top; - empty-cells: show; -} - - -/* style for "other" displays */ -table.otherinfo { - border-spacing: 0px; - border-collapse: separate; - width: 100%; -} - -table.otherinfo th.header{ - padding-top: 10px; - border-bottom: 1px solid gray; - font-weight: bold; - background-color: white; - color: #707040; -} - -table.otherinfo th { - border-bottom: 1px solid #afafaf; - font-weight: bold; - text-align: left; -}
--- a/roundup/templates/classic/html/user.index Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,35 +0,0 @@ -<!-- dollarId: user.index,v 1.3 2002/07/09 05:29:51 richard Exp dollar--> -<tal:block metal:use-macro="templates/page/macros/icing"> -<title metal:fill-slot="head_title">User listing</title> -<td class="page-header-top" metal:fill-slot="body_title"> - <h2>User listing</h2> -</td> -<td class="content" metal:fill-slot="content"> - -<span tal:condition="not:context/is_view_ok"> -You are not allowed to view this page. -</span> - -<table width="100%" tal:condition="context/is_view_ok" class="list"> -<tr> - <th>Username</th> - <th>Real name</th> - <th>Organisation</th> - <th>Email address</th> - <th>Phone number</th> -</tr> -<tr tal:repeat="user context/list" - tal:attributes="class python:['row-normal', 'row-alt'][repeat['user'].even()]"> - <td> - <a tal:attributes="href string:user${user/id}" - tal:content="user/username">username</a> - </td> - <td tal:content="user/realname">realname</td> - <td tal:content="user/organisation">organisation</td> - <td tal:content="python:user.address.email()">address</td> - <td tal:content="user/phone">phone</td> -</tr> -</table> -</td> - -</tal:block>
--- a/roundup/templates/classic/html/user.item Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,105 +0,0 @@ -<!-- dollarId: user.item,v 1.7 2002/08/16 04:29:04 richard Exp dollar--> -<tal:block metal:use-macro="templates/page/macros/icing"> -<title metal:fill-slot="head_title">User editing</title> -<td class="page-header-top" metal:fill-slot="body_title"> - <h2>User editing</h2> -</td> -<td class="content" metal:fill-slot="content"> -<span tal:condition="python:not (context.is_view_ok() or context.is_edit_ok())"> -You are not allowed to view this page. -</span> - -<form method="POST" onSubmit="return submit_once()" - enctype="multipart/form-data" tal:condition="context/is_edit_ok"> - -<input type="hidden" name=":required" value="username,address"> - -<table class="form"> - <tr> - <th>Name</th> - <td tal:content="structure context/realname/field">realname</td> - </tr> - <tr> - <th>Login Name</th> - <td tal:content="structure context/username/field">username</td> - </tr> - <tr> - <th>Login Password</th> - <td tal:content="structure context/password/field">password</td> - </tr> - <tr> - <th>Confirm Password</th> - <td tal:content="structure context/password/confirm">password</td> - </tr> - <tr tal:condition="python:request.user.hasPermission('Web Roles')"> - <th>Roles</th> - <td tal:condition="context/id" - tal:content="structure context/roles/field">roles</td> - <td tal:condition="not:context/id"> - <input name="roles" tal:attributes="value db/config/NEW_WEB_USER_ROLES"> - </td> - </tr> - <tr> - <th>Phone</th> - <td tal:content="structure context/phone/field">phone</td> - </tr> - <tr> - <th>Organisation</th> - <td tal:content="structure context/organisation/field">organisation</td> - </tr> - <tr> - <th>E-mail address</th> - <td tal:content="structure context/address/field">address</td> - </tr> - <tr> - <th>Alternate E-mail addresses<br>One address per line</th> - <td tal:content="structure context/alternate_addresses/multiline">alternate_addresses</td> - </tr> - - <tr> - <td> </td> - <td tal:content="structure context/submit">submit button here</td> - </tr> -</table> -</form> - -<table class="otherinfo" tal:condition="context/queries"> - <tr><th colspan="2" class="header">Queries</th></tr> - <tr><th>Name</th><th>Display</th></tr> - <tr tal:repeat="query context/queries"> - <td><a tal:attributes="href string:query${query/id}" - tal:content="query/name"></a></td> - <td> - <a tal:attributes="href python:'%s%s'%(query['klass'], query['url'])">display</a> - </td> - </tr> -</table> - -<table class="form" tal:condition="context/is_only_view_ok"> - <tr> - <th colspan=2 class="header" tal:content="context/realname">realname</th> - </tr> - <tr> - <th>Login Name</th> - <td tal:content="context/username">username</td> - </tr> - <tr> - <th>Phone</th> - <td tal:content="context/phone">phone</td> - </tr> - <tr> - <th>Organisation</th> - <td tal:content="context/organisation">organisation</td> - </tr> - <tr> - <th>E-mail address</th> - <td tal:content="context/address/email">address</td> - </tr> -</table> - -<tal:block tal:condition="python:context.id and context.is_view_ok()" - tal:replace="structure context/history" /> - -</td> - -</tal:block>
--- a/roundup/templates/classic/interfaces.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,35 +0,0 @@ -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: interfaces.py,v 1.15 2002-09-09 23:55:19 richard Exp $ - -from roundup import mailgw -from roundup.cgi import client - -class Client(client.Client): - ''' derives basic CGI implementation from the standard module, - with any specific extensions - ''' - pass - -class MailGW(mailgw.MailGW): - ''' derives basic mail gateway implementation from the standard module, - with any specific extensions - ''' - pass - -# vim: set filetype=python ts=4 sw=4 et si
--- a/roundup/templates/minimal/config.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,92 +0,0 @@ -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: config.py,v 1.1 2002-09-26 04:15:06 richard Exp $ - -import os - -# roundup home is this package's directory -TRACKER_HOME=os.path.split(__file__)[0] - -# The SMTP mail host that roundup will use to send mail -MAILHOST = 'localhost' - -# The domain name used for email addresses. -MAIL_DOMAIN = 'your.tracker.email.domain.example' - -# This is the directory that the database is going to be stored in -DATABASE = os.path.join(TRACKER_HOME, 'db') - -# This is the directory that the HTML templates reside in -TEMPLATES = os.path.join(TRACKER_HOME, 'html') - -# A descriptive name for your roundup instance -TRACKER_NAME = 'Roundup issue tracker' - -# The email address that mail to roundup should go to -TRACKER_EMAIL = 'issue_tracker@%s'%MAIL_DOMAIN - -# The web address that the instance is viewable at -TRACKER_WEB = 'http://your.tracker.url.example/' - -# The email address that roundup will complain to if it runs into trouble -ADMIN_EMAIL = 'roundup-admin@%s'%MAIL_DOMAIN - -# Where to place the web filtering HTML on the index page -FILTER_POSITION = 'bottom' # one of 'top', 'bottom', 'top and bottom' - -# -# SECURITY DEFINITIONS -# -# define the Roles that a user gets when they register with the tracker -# these are a comma-separated string of role names (e.g. 'Admin,User') -NEW_WEB_USER_ROLES = 'User' -NEW_EMAIL_USER_ROLES = 'User' - -# Send nosy messages to the author of the message -MESSAGES_TO_AUTHOR = 'no' # either 'yes' or 'no' - -# Does the author of a message get placed on the nosy list automatically? -# If 'new' is used, then the author will only be added when a message -# creates a new issue. If 'yes', then the author will be added on followups -# too. If 'no', they're never added to the nosy. -ADD_AUTHOR_TO_NOSY = 'new' # one of 'yes', 'no', 'new' - -# Do the recipients (To:, Cc:) of a message get placed on the nosy list? -# If 'new' is used, then the recipients will only be added when a message -# creates a new issue. If 'yes', then the recipients will be added on followups -# too. If 'no', they're never added to the nosy. -ADD_RECIPIENTS_TO_NOSY = 'new' # either 'yes', 'no', 'new' - -# Where to place the email signature -EMAIL_SIGNATURE_POSITION = 'bottom' # one of 'top', 'bottom', 'none' - -# Keep email citations when accepting messages. Setting this to "no" strips -# out "quoted" text from the message. Signatures are also stripped. -EMAIL_KEEP_QUOTED_TEXT = 'yes' # either 'yes' or 'no' - -# Preserve the email body as is - that is, keep the citations _and_ -# signatures. -EMAIL_LEAVE_BODY_UNCHANGED = 'no' # either 'yes' or 'no' - -# Default class to use in the mailgw if one isn't supplied in email -# subjects. To disable, comment out the variable below or leave it blank. -# Examples: -MAIL_DEFAULT_CLASS = 'issue' # use "issue" class by default -#MAIL_DEFAULT_CLASS = '' # disable (or just comment the var out) - -# vim: set filetype=python ts=4 sw=4 et si
--- a/roundup/templates/minimal/dbinit.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,106 +0,0 @@ -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: dbinit.py,v 1.1 2002-09-26 04:15:07 richard Exp $ - -import os - -import config -from select_db import Database, Class, FileClass, IssueClass - -def open(name=None): - ''' as from the roundupdb method openDB - ''' - from roundup.hyperdb import String, Password, Date, Link, Multilink - - # open the database - db = Database(config, name) - - # - # Now initialise the schema. Must do this each time the database is - # opened. - # - - # The "Minimal" template gets only one class, the required "user" - # class. That's it. And even that has the bare minimum of properties. - - # Note: roles is a comma-separated string of Role names - user = Class(db, "user", username=String(), password=Password(), - address=String(), alternate_addresses=String(), roles=String()) - user.setkey("username") - - # - # SECURITY SETTINGS - # - # new permissions for this schema - for cl in ('user', ): - db.security.addPermission(name="Edit", klass=cl, - description="User is allowed to edit "+cl) - db.security.addPermission(name="View", klass=cl, - description="User is allowed to access "+cl) - - # and give the regular users access to the web and email interface - p = db.security.getPermission('Web Access') - db.security.addPermissionToRole('User', p) - p = db.security.getPermission('Email Access') - db.security.addPermissionToRole('User', p) - - # May users view other user information? Comment these lines out - # if you don't want them to - p = db.security.getPermission('View', 'user') - db.security.addPermissionToRole('User', p) - - # Assign the appropriate permissions to the anonymous user's Anonymous - # Role. Choices here are: - # - Allow anonymous users to register through the web - p = db.security.getPermission('Web Registration') - db.security.addPermissionToRole('Anonymous', p) - # - Allow anonymous (new) users to register through the email gateway - p = db.security.getPermission('Email Registration') - db.security.addPermissionToRole('Anonymous', p) - - import detectors - detectors.init(db) - - # schema is set up - run any post-initialisation - db.post_init() - return db - -def init(adminpw): - ''' as from the roundupdb method initDB - - Open the new database, and add new nodes - used for initialisation. You - can edit this before running the "roundup-admin initialise" command to - change the initial database entries. - ''' - dbdir = os.path.join(config.DATABASE, 'files') - if not os.path.isdir(dbdir): - os.makedirs(dbdir) - - db = open("admin") - db.clear() - - # create the two default users - user = db.getclass('user') - user.create(username="admin", password=adminpw, - address=config.ADMIN_EMAIL, roles='Admin') - user.create(username="anonymous", roles='Anonymous') - - db.commit() - -# vim: set filetype=python ts=4 sw=4 et si -
--- a/roundup/templates/minimal/html/style.css Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,305 +0,0 @@ -/* main page styles */ -body.body { - font-family: sans-serif, Arial, Helvetica; - color: #333333; -} -a[href]:hover { color:blue; text-decoration: underline; } -a[href]:link { color:blue; text-decoration: none; } -a[href] { color:blue; text-decoration: none; } - -table.body { - border: 0; - padding: 0; - border-spacing: 0px; - border-collapse: separate; -} - -td.page-header-left { - padding: 5px; - border-bottom: 1px solid #444444; -} - -td.page-header-top { - border-bottom: 1px solid #444444; - padding: 5px; -} - -td.sidebar { - padding: 1 0 0 1; -} - -td.sidebar p.classblock { - padding: 0 5 0 5; - margin: 1 1 1 1; - border: 1px solid #444444; - background-color: #eeeeee; -} - -td.sidebar p.userblock { - padding: 0 5 0 5; - margin: 1 1 1 1; - border: 1px solid #444444; - background-color: #eeeeff; -} - -td.content { - padding: 1 5 1 5; - vertical-align: top; -} - -p.ok-message { - background-color: #22bb22; - padding: 5 5 5 5; - color: white; - font-weight: bold; -} -p.error-message { - background-color: #bb2222; - padding: 5 5 5 5; - color: white; - font-weight: bold; -} - - -/* style for forms */ -table.form { - padding: 2; - border-spacing: 0px; - border-collapse: separate; -} - -table.form th { - font-weight: bold; - color: #333388; - text-align: right; - vertical-align: top; -} - -table.form th.header { - font-weight: bold; - color: #333388; - background-color: #eeeeff; - text-align: left; -} - -table.form td.optional { - font-weight: bold; - font-style: italic; - color: #333333; - empty-cells: show; -} - -table.form td { - color: #333333; - empty-cells: show; -} - -table.form td.html { - color: #777777; - empty-cells: show; -} - -/* style for lists */ -table.list { - border-spacing: 0px; - border-collapse: separate; - width: 100%; -} - -table.list th { - padding: 0 4 0 4; - color: #404070; - background-color: #eeeeff; - border-right: 1px solid #404070; - border-top: 1px solid #404070; - border-bottom: 1px solid #404070; - vertical-align: top; -} -table.list th a[href]:hover { color: #404070 } -table.list th a[href]:link { color: #404070 } -table.list th a[href] { color: #404070 } -table.list th.group { - background-color: #f4f4ff; - text-align: center; -} - -table.list td { - padding: 0 4 0 4; - border: 0 2 0 2; - border-right: 1px solid #404070; - color: #404070; - background-color: white; - vertical-align: top; -} - -table.list tr.normal td { - empty-cells: show; -} - -table.list tr.alt td { - background-color: #efefef; - empty-cells: show; -} - -table.list td:first-child { - border-left: 1px solid #404070; - border-right: 1px solid #404070; - empty-cells: show; -} - -table.list th:first-child { - border-left: 1px solid #404070; - border-right: 1px solid #404070; - empty-cells: show; -} - - -/* style for message displays */ -table.messages { - border-spacing: 0px; - border-collapse: separate; - width: 100%; -} - -table.messages th.header{ - padding-top: 10px; - border-bottom: 1px solid gray; - font-weight: bold; - background-color: white; - color: #707040; -} - -table.messages th { - font-weight: bold; - color: black; - text-align: left; -} - -table.messages td { - font-family: monospace; - background-color: #efefef; - border-top: 1px solid #afafaf; - border-bottom: 1px solid #afafaf; - color: black; - empty-cells: show; -} - -/* style for file displays */ -table.files { - border-spacing: 0px; - border-collapse: separate; - width: 100%; -} - -table.files th.header{ - padding-top: 10px; - border-bottom: 1px solid gray; - font-weight: bold; - background-color: white; - color: #707040; -} - -table.files th { - border-bottom: 1px solid #afafaf; - font-weight: bold; - text-align: left; -} - -table.files td { - font-family: monospace; - empty-cells: show; -} - -/* style for history displays */ -table.history { - border-spacing: 0px; - border-collapse: separate; - width: 100%; -} - -table.history th.header{ - padding-top: 10px; - border-bottom: 1px solid gray; - font-weight: bold; - background-color: white; - color: #707040; - font-size: 100%; -} - -table.history th { - border-bottom: 1px solid #afafaf; - font-weight: bold; - text-align: left; - font-size: 90%; -} - -table.history td { - font-size: 90%; - vertical-align: top; - empty-cells: show; -} - - -/* style for class list */ -table.classlist { - border-spacing: 0px; - border-collapse: separate; - width: 100%; -} - -table.classlist th.header{ - padding-top: 10px; - border-bottom: 1px solid gray; - font-weight: bold; - background-color: white; - color: #707040; -} - -table.classlist th { - font-weight: bold; - text-align: left; -} - - -/* style for class help display */ -table.classhelp { - border-spacing: 0px; - border-collapse: separate; - width: 100%; -} - -table.classhelp th { - font-weight: bold; - text-align: left; - color: #707040; -} - -table.classhelp td { - padding: 2 2 2 2; - border: 1px solid black; - text-align: left; - vertical-align: top; - empty-cells: show; -} - - -/* style for "other" displays */ -table.otherinfo { - border-spacing: 0px; - border-collapse: separate; - width: 100%; -} - -table.otherinfo th.header{ - padding-top: 10px; - border-bottom: 1px solid gray; - font-weight: bold; - background-color: white; - color: #707040; -} - -table.otherinfo th { - border-bottom: 1px solid #afafaf; - font-weight: bold; - text-align: left; -}
--- a/roundup/templates/minimal/html/user.index Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,29 +0,0 @@ -<!-- dollarId: user.index,v 1.3 2002/07/09 05:29:51 richard Exp dollar--> -<tal:block metal:use-macro="templates/page/macros/icing"> -<title metal:fill-slot="head_title">User listing</title> -<td class="page-header-top" metal:fill-slot="body_title"> - <h2>User listing</h2> -</td> -<td class="content" metal:fill-slot="content"> - -<span tal:condition="not:context/is_view_ok"> -You are not allowed to view this page. -</span> - -<table width="100%" tal:condition="context/is_view_ok" class="list"> -<tr> - <th>Username</th> - <th>Email address</th> -</tr> -<tr tal:repeat="user context/list" - tal:attributes="class python:['row-normal', 'row-alt'][repeat['user'].even()]"> - <td> - <a tal:attributes="href string:user${user/id}" - tal:content="user/username">username</a> - </td> - <td tal:content="python:user.address.email()">address</td> -</tr> -</table> -</td> - -</tal:block>
--- a/roundup/templates/minimal/interfaces.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,35 +0,0 @@ -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: interfaces.py,v 1.1 2002-09-26 04:15:07 richard Exp $ - -from roundup import mailgw -from roundup.cgi import client - -class Client(client.Client): - ''' derives basic CGI implementation from the standard module, - with any specific extensions - ''' - pass - -class MailGW(mailgw.MailGW): - ''' derives basic mail gateway implementation from the standard module, - with any specific extensions - ''' - pass - -# vim: set filetype=python ts=4 sw=4 et si
--- a/setup.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,193 +0,0 @@ -#! /usr/bin/env python -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: setup.py,v 1.39 2002-09-23 08:17:50 richard Exp $ - -from distutils.core import setup, Extension -from distutils.util import get_platform -from distutils.command.build_scripts import build_scripts - -import sys, os, string -from glob import glob - -from roundup.templates.builder import makeHtmlBase - - -############################################################################# -### Build script files -############################################################################# - -class build_scripts_create(build_scripts): - """ Overload the build_scripts command and create the scripts - from scratch, depending on the target platform. - - You have to define the name of your package in an inherited - class (due to the delayed instantiation of command classes - in distutils, this cannot be passed to __init__). - - The scripts are created in an uniform scheme: they start the - run() function in the module - - <packagename>.scripts.<mangled_scriptname> - - The mangling of script names replaces '-' and '/' characters - with '-' and '.', so that they are valid module paths. - """ - package_name = None - - def copy_scripts(self): - """ Create each script listed in 'self.scripts' - """ - if not self.package_name: - raise Exception("You have to inherit build_scripts_create and" - " provide a package name") - - to_module = string.maketrans('-/', '_.') - - self.mkpath(self.build_dir) - for script in self.scripts: - outfile = os.path.join(self.build_dir, os.path.basename(script)) - - #if not self.force and not newer(script, outfile): - # self.announce("not copying %s (up-to-date)" % script) - # continue - - if self.dry_run: - self.announce("would create %s" % outfile) - continue - - module = os.path.splitext(os.path.basename(script))[0] - module = string.translate(module, to_module) - script_vars = { - 'python': os.path.normpath(sys.executable), - 'package': self.package_name, - 'module': module, - } - - self.announce("creating %s" % outfile) - file = open(outfile, 'w') - - try: - if sys.platform == "win32": - file.write('@echo off\n' - 'if NOT "%%_4ver%%" == "" %(python)s -O -c "from %(package)s.scripts.%(module)s import run; run()" %%$\n' - 'if "%%_4ver%%" == "" %(python)s -O -c "from %(package)s.scripts.%(module)s import run; run()" %%*\n' - % script_vars) - else: - file.write('#! %(python)s -O\n' - 'from %(package)s.scripts.%(module)s import run\n' - 'run()\n' - % script_vars) - finally: - file.close() - os.chmod(outfile, 0755) - - -class build_scripts_roundup(build_scripts_create): - package_name = 'roundup' - - -def scriptname(path): - """ Helper for building a list of script names from a list of - module files. - """ - script = os.path.splitext(os.path.basename(path))[0] - script = string.replace(script, '_', '-') - if sys.platform == "win32": - script = script + ".bat" - return script - - - -############################################################################# -### Main setup stuff -############################################################################# - -def isTemplateDir(dir): - return dir[0] != '.' and dir != 'CVS' and os.path.isdir(dir) \ - and os.path.isfile(os.path.join(dir, '__init__.py')) - -# use that function to list all the templates -templates = map(os.path.basename, filter(isTemplateDir, - glob(os.path.join('roundup', 'templates', '*')))) - -def buildTemplates(): - for template in templates: - tdir = os.path.join('roundup', 'templates', template) - makeHtmlBase(tdir) - -if __name__ == '__main__': - # build list of scripts from their implementation modules - roundup_scripts = map(scriptname, glob('roundup/scripts/[!_]*.py')) - - # template munching - templates = map(os.path.basename, filter(isTemplateDir, - glob(os.path.join('roundup', 'templates', '*')))) - packagelist = [ - 'roundup', - 'roundup.cgi', - 'roundup.cgi.PageTemplates', - 'roundup.cgi.TAL', - 'roundup.cgi.ZTUtils', - 'roundup.backends', - 'roundup.scripts', - 'roundup.templates' - ] - installdatafiles = [ - ('share/roundup/cgi-bin', ['cgi-bin/roundup.cgi']), - ] - - # munge the template HTML into the htmlbase module - buildTemplates() - - # add the templates to the setup packages and data files lists - for template in templates: - tdir = os.path.join('roundup', 'templates', template) - - # add the template package and subpackage - packagelist.append('roundup.templates.%s' % template) - packagelist.append('roundup.templates.%s.detectors' % template) - - # scan for data files - tfiles = glob(os.path.join(tdir, 'html', '*')) - tfiles = filter(os.path.isfile, tfiles) - installdatafiles.append( - ('share/roundup/templates/%s/html' % template, tfiles) - ) - - # perform the setup action - from roundup import __version__ - setup( - name = "roundup", - version = __version__, - description = "Roundup issue tracking system.", - author = "Richard Jones", - author_email = "richard@users.sourceforge.net", - url = 'http://sourceforge.net/projects/roundup/', - packages = packagelist, - - # Override certain command classes with our own ones - cmdclass = { - 'build_scripts': build_scripts_roundup, - }, - scripts = roundup_scripts, - - data_files = installdatafiles - ) - -# vim: set filetype=python ts=4 sw=4 et si
--- a/test/test_dates.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,155 +0,0 @@ -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: test_dates.py,v 1.13 2002-09-10 00:19:54 richard Exp $ - -import unittest, time - -from roundup.date import Date, Interval - -class DateTestCase(unittest.TestCase): - def testDateInterval(self): - ae = self.assertEqual - date = Date("2000-06-26.00:34:02 + 2d") - ae(str(date), '2000-06-28.00:34:02') - date = Date("2000-02-27 + 2d") - ae(str(date), '2000-02-29.00:00:00') - date = Date("2001-02-27 + 2d") - ae(str(date), '2001-03-01.00:00:00') - - def testDate(self): - ae = self.assertEqual - date = Date("2000-04-17") - ae(str(date), '2000-04-17.00:00:00') - date = Date("2000-4-7") - ae(str(date), '2000-04-07.00:00:00') - date = Date("2000-4-17") - ae(str(date), '2000-04-17.00:00:00') - date = Date("01-25") - y, m, d, x, x, x, x, x, x = time.gmtime(time.time()) - ae(str(date), '%s-01-25.00:00:00'%y) - date = Date("2000-04-17.03:45") - ae(str(date), '2000-04-17.03:45:00') - date = Date("08-13.22:13") - ae(str(date), '%s-08-13.22:13:00'%y) - date = Date("11-07.09:32:43") - ae(str(date), '%s-11-07.09:32:43'%y) - date = Date("14:25") - ae(str(date), '%s-%02d-%02d.14:25:00'%(y, m, d)) - date = Date("8:47:11") - ae(str(date), '%s-%02d-%02d.08:47:11'%(y, m, d)) - - def testOffset(self): - ae = self.assertEqual - date = Date("2000-04-17", -5) - ae(str(date), '2000-04-17.00:00:00') - date = Date("01-25", -5) - y, m, d, x, x, x, x, x, x = time.gmtime(time.time()) - ae(str(date), '%s-01-25.00:00:00'%y) - date = Date("2000-04-17.03:45", -5) - ae(str(date), '2000-04-17.08:45:00') - date = Date("08-13.22:13", -5) - ae(str(date), '%s-08-14.03:13:00'%y) - date = Date("11-07.09:32:43", -5) - ae(str(date), '%s-11-07.14:32:43'%y) - date = Date("14:25", -5) - ae(str(date), '%s-%02d-%02d.19:25:00'%(y, m, d)) - date = Date("8:47:11", -5) - ae(str(date), '%s-%02d-%02d.13:47:11'%(y, m, d)) - - # now check calculations - date = Date('2000-01-01') + Interval('- 2y 2m') - ae(str(date), '1997-11-01.00:00:00') - date = Date('2000-01-01 - 2y 2m') - ae(str(date), '1997-11-01.00:00:00') - date = Date('2000-01-01') + Interval('2m') - ae(str(date), '2000-03-01.00:00:00') - date = Date('2000-01-01 + 2m') - ae(str(date), '2000-03-01.00:00:00') - - date = Date('2000-01-01') + Interval('60d') - ae(str(date), '2000-03-01.00:00:00') - date = Date('2001-01-01') + Interval('60d') - ae(str(date), '2001-03-02.00:00:00') - - # time additions - date = Date('2000-02-28.23:59:59') + Interval('00:00:01') - ae(str(date), '2000-02-29.00:00:00') - date = Date('2001-02-28.23:59:59') + Interval('00:00:01') - ae(str(date), '2001-03-01.00:00:00') - - date = Date('2000-02-28.23:58:59') + Interval('00:01:01') - ae(str(date), '2000-02-29.00:00:00') - date = Date('2001-02-28.23:58:59') + Interval('00:01:01') - ae(str(date), '2001-03-01.00:00:00') - - date = Date('2000-02-28.22:58:59') + Interval('01:01:01') - ae(str(date), '2000-02-29.00:00:00') - date = Date('2001-02-28.22:58:59') + Interval('01:01:01') - ae(str(date), '2001-03-01.00:00:00') - - date = Date('2000-02-28.22:58:59') + Interval('00:00:3661') - ae(str(date), '2000-02-29.00:00:00') - date = Date('2001-02-28.22:58:59') + Interval('00:00:3661') - ae(str(date), '2001-03-01.00:00:00') - - # now subtractions - date = Date('2000-01-01') - Interval('- 2y 2m') - ae(str(date), '2002-03-01.00:00:00') - date = Date('2000-01-01') - Interval('2m') - ae(str(date), '1999-11-01.00:00:00') - - date = Date('2000-03-01') - Interval('60d') - ae(str(date), '2000-01-01.00:00:00') - date = Date('2001-03-02') - Interval('60d') - ae(str(date), '2001-01-01.00:00:00') - - date = Date('2000-02-29.00:00:00') - Interval('00:00:01') - ae(str(date), '2000-02-28.23:59:59') - date = Date('2001-03-01.00:00:00') - Interval('00:00:01') - ae(str(date), '2001-02-28.23:59:59') - - date = Date('2000-02-29.00:00:00') - Interval('00:01:01') - ae(str(date), '2000-02-28.23:58:59') - date = Date('2001-03-01.00:00:00') - Interval('00:01:01') - ae(str(date), '2001-02-28.23:58:59') - - date = Date('2000-02-29.00:00:00') - Interval('01:01:01') - ae(str(date), '2000-02-28.22:58:59') - date = Date('2001-03-01.00:00:00') - Interval('01:01:01') - ae(str(date), '2001-02-28.22:58:59') - - date = Date('2000-02-29.00:00:00') - Interval('00:00:3661') - ae(str(date), '2000-02-28.22:58:59') - date = Date('2001-03-01.00:00:00') - Interval('00:00:3661') - ae(str(date), '2001-02-28.22:58:59') - - def testInterval(self): - ae = self.assertEqual - ae(str(Interval('3y')), '+ 3y') - ae(str(Interval('2 y 1 m')), '+ 2y 1m') - ae(str(Interval('1m 25d')), '+ 1m 25d') - ae(str(Interval('-2w 3 d ')), '- 17d') - ae(str(Interval(' - 1 d 2:50 ')), '- 1d 2:50') - ae(str(Interval(' 14:00 ')), '+ 14:00') - ae(str(Interval(' 0:04:33 ')), '+ 0:04:33') - -def suite(): - return unittest.makeSuite(DateTestCase, 'test') - - -# vim: set filetype=python ts=4 sw=4 et si
--- a/test/test_db.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,795 +0,0 @@ -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: test_db.py,v 1.56 2002-09-26 03:04:24 richard Exp $ - -import unittest, os, shutil, time - -from roundup.hyperdb import String, Password, Link, Multilink, Date, \ - Interval, DatabaseError, Boolean, Number -from roundup import date, password -from roundup.indexer import Indexer - -def setupSchema(db, create, module): - status = module.Class(db, "status", name=String()) - status.setkey("name") - user = module.Class(db, "user", username=String(), password=Password(), - assignable=Boolean(), age=Number(), roles=String()) - user.setkey("username") - file = module.FileClass(db, "file", name=String(), type=String(), - comment=String(indexme="yes"), fooz=Password()) - issue = module.IssueClass(db, "issue", title=String(indexme="yes"), - status=Link("status"), nosy=Multilink("user"), deadline=Date(), - foo=Interval(), files=Multilink("file"), assignedto=Link('user')) - session = module.Class(db, 'session', title=String()) - session.disableJournalling() - db.post_init() - if create: - user.create(username="admin", roles='Admin') - status.create(name="unread") - status.create(name="in-progress") - status.create(name="testing") - status.create(name="resolved") - db.commit() - -class MyTestCase(unittest.TestCase): - def tearDown(self): - self.db.close() - if hasattr(self, 'db2'): - self.db2.close() - if os.path.exists('_test_dir'): - shutil.rmtree('_test_dir') - -class config: - DATABASE='_test_dir' - MAILHOST = 'localhost' - MAIL_DOMAIN = 'fill.me.in.' - TRACKER_NAME = 'Roundup issue tracker' - TRACKER_EMAIL = 'issue_tracker@%s'%MAIL_DOMAIN - TRACKER_WEB = 'http://some.useful.url/' - ADMIN_EMAIL = 'roundup-admin@%s'%MAIL_DOMAIN - FILTER_POSITION = 'bottom' # one of 'top', 'bottom', 'top and bottom' - ANONYMOUS_ACCESS = 'deny' # either 'deny' or 'allow' - ANONYMOUS_REGISTER = 'deny' # either 'deny' or 'allow' - MESSAGES_TO_AUTHOR = 'no' # either 'yes' or 'no' - EMAIL_SIGNATURE_POSITION = 'bottom' - -class anydbmDBTestCase(MyTestCase): - def setUp(self): - from roundup.backends import anydbm - # remove previous test, ignore errors - if os.path.exists(config.DATABASE): - shutil.rmtree(config.DATABASE) - os.makedirs(config.DATABASE + '/files') - self.db = anydbm.Database(config, 'admin') - setupSchema(self.db, 1, anydbm) - self.db2 = anydbm.Database(config, 'admin') - setupSchema(self.db2, 0, anydbm) - - def testStringChange(self): - # test set & retrieve - self.db.issue.create(title="spam", status='1') - self.assertEqual(self.db.issue.get('1', 'title'), 'spam') - - # change and make sure we retrieve the correct value - self.db.issue.set('1', title='eggs') - self.assertEqual(self.db.issue.get('1', 'title'), 'eggs') - - # do some commit stuff - self.db.commit() - self.assertEqual(self.db.issue.get('1', 'title'), 'eggs') - self.db.issue.create(title="spam", status='1') - self.db.commit() - self.assertEqual(self.db.issue.get('2', 'title'), 'spam') - self.db.issue.set('2', title='ham') - self.assertEqual(self.db.issue.get('2', 'title'), 'ham') - self.db.commit() - self.assertEqual(self.db.issue.get('2', 'title'), 'ham') - - # make sure we can unset - self.db.issue.set('1', title=None) - self.assertEqual(self.db.issue.get('1', "title"), None) - - def testLinkChange(self): - self.db.issue.create(title="spam", status='1') - self.assertEqual(self.db.issue.get('1', "status"), '1') - self.db.issue.set('1', status='2') - self.assertEqual(self.db.issue.get('1', "status"), '2') - self.db.issue.set('1', status=None) - self.assertEqual(self.db.issue.get('1', "status"), None) - - def testMultilinkChange(self): - u1 = self.db.user.create(username='foo') - u2 = self.db.user.create(username='bar') - self.db.issue.create(title="spam", nosy=[u1]) - self.assertEqual(self.db.issue.get('1', "nosy"), [u1]) - self.db.issue.set('1', nosy=[]) - self.assertEqual(self.db.issue.get('1', "nosy"), []) - self.db.issue.set('1', nosy=[u1,u2]) - self.assertEqual(self.db.issue.get('1', "nosy"), [u1,u2]) - - def testDateChange(self): - self.db.issue.create(title="spam", status='1') - a = self.db.issue.get('1', "deadline") - self.db.issue.set('1', deadline=date.Date()) - b = self.db.issue.get('1', "deadline") - self.db.commit() - self.assertNotEqual(a, b) - self.assertNotEqual(b, date.Date('1970-1-1 00:00:00')) - self.db.issue.set('1', deadline=date.Date()) - self.db.issue.set('1', deadline=None) - self.assertEqual(self.db.issue.get('1', "deadline"), None) - - def testIntervalChange(self): - self.db.issue.create(title="spam", status='1') - a = self.db.issue.get('1', "foo") - self.db.issue.set('1', foo=date.Interval('-1d')) - self.assertNotEqual(self.db.issue.get('1', "foo"), a) - self.db.issue.set('1', foo=None) - self.assertEqual(self.db.issue.get('1', "foo"), None) - - def testBooleanChange(self): - userid = self.db.user.create(username='foo', assignable=1) - self.assertEqual(1, self.db.user.get(userid, 'assignable')) - self.db.user.set(userid, assignable=0) - self.assertEqual(self.db.user.get(userid, 'assignable'), 0) - self.db.user.set(userid, assignable=None) - self.assertEqual(self.db.user.get('1', "assignable"), None) - - def testNumberChange(self): - nid = self.db.user.create(username='foo', age=1) - self.assertEqual(1, self.db.user.get(nid, 'age')) - self.db.user.set('1', age=3) - self.assertNotEqual(self.db.user.get('1', 'age'), 1) - self.db.user.set('1', age=1.0) - self.db.user.set('1', age=None) - self.assertEqual(self.db.user.get('1', "age"), None) - - def testKeyValue(self): - newid = self.db.user.create(username="spam") - self.assertEqual(self.db.user.lookup('spam'), newid) - self.db.commit() - self.assertEqual(self.db.user.lookup('spam'), newid) - self.db.user.retire(newid) - self.assertRaises(KeyError, self.db.user.lookup, 'spam') - - def testNewProperty(self): - self.db.issue.create(title="spam", status='1') - self.db.issue.addprop(fixer=Link("user")) - # force any post-init stuff to happen - self.db.post_init() - props = self.db.issue.getprops() - keys = props.keys() - keys.sort() - self.assertEqual(keys, ['activity', 'assignedto', 'creation', - 'creator', 'deadline', 'files', 'fixer', 'foo', 'id', 'messages', - 'nosy', 'status', 'superseder', 'title']) - self.assertEqual(self.db.issue.get('1', "fixer"), None) - - def testRetire(self): - self.db.issue.create(title="spam", status='1') - b = self.db.status.get('1', 'name') - a = self.db.status.list() - self.db.status.retire('1') - # make sure the list is different - self.assertNotEqual(a, self.db.status.list()) - # can still access the node if necessary - self.assertEqual(self.db.status.get('1', 'name'), b) - self.db.commit() - self.assertEqual(self.db.status.get('1', 'name'), b) - self.assertNotEqual(a, self.db.status.list()) - - def testSerialisation(self): - self.db.issue.create(title="spam", status='1', - deadline=date.Date(), foo=date.Interval('-1d')) - self.db.commit() - assert isinstance(self.db.issue.get('1', 'deadline'), date.Date) - assert isinstance(self.db.issue.get('1', 'foo'), date.Interval) - self.db.user.create(username="fozzy", - password=password.Password('t. bear')) - self.db.commit() - assert isinstance(self.db.user.get('1', 'password'), password.Password) - - def testTransactions(self): - # remember the number of items we started - num_issues = len(self.db.issue.list()) - num_files = self.db.numfiles() - self.db.issue.create(title="don't commit me!", status='1') - self.assertNotEqual(num_issues, len(self.db.issue.list())) - self.db.rollback() - self.assertEqual(num_issues, len(self.db.issue.list())) - self.db.issue.create(title="please commit me!", status='1') - self.assertNotEqual(num_issues, len(self.db.issue.list())) - self.db.commit() - self.assertNotEqual(num_issues, len(self.db.issue.list())) - self.db.rollback() - self.assertNotEqual(num_issues, len(self.db.issue.list())) - self.db.file.create(name="test", type="text/plain", content="hi") - self.db.rollback() - self.assertEqual(num_files, self.db.numfiles()) - for i in range(10): - self.db.file.create(name="test", type="text/plain", - content="hi %d"%(i)) - self.db.commit() - num_files2 = self.db.numfiles() - self.assertNotEqual(num_files, num_files2) - self.db.file.create(name="test", type="text/plain", content="hi") - self.db.rollback() - self.assertNotEqual(num_files, self.db.numfiles()) - self.assertEqual(num_files2, self.db.numfiles()) - - def testDestroyNoJournalling(self): - self.innerTestDestroy(klass=self.db.session) - - def testDestroyJournalling(self): - self.innerTestDestroy(klass=self.db.issue) - - def innerTestDestroy(self, klass): - newid = klass.create(title='Mr Friendly') - n = len(klass.list()) - self.assertEqual(klass.get(newid, 'title'), 'Mr Friendly') - klass.destroy(newid) - self.assertRaises(IndexError, klass.get, newid, 'title') - self.assertNotEqual(len(klass.list()), n) - if klass.do_journal: - self.assertRaises(IndexError, klass.history, newid) - - # now with a commit - newid = klass.create(title='Mr Friendly') - n = len(klass.list()) - self.assertEqual(klass.get(newid, 'title'), 'Mr Friendly') - self.db.commit() - klass.destroy(newid) - self.assertRaises(IndexError, klass.get, newid, 'title') - self.db.commit() - self.assertRaises(IndexError, klass.get, newid, 'title') - self.assertNotEqual(len(klass.list()), n) - if klass.do_journal: - self.assertRaises(IndexError, klass.history, newid) - - # now with a rollback - newid = klass.create(title='Mr Friendly') - n = len(klass.list()) - self.assertEqual(klass.get(newid, 'title'), 'Mr Friendly') - self.db.commit() - klass.destroy(newid) - self.assertNotEqual(len(klass.list()), n) - self.assertRaises(IndexError, klass.get, newid, 'title') - self.db.rollback() - self.assertEqual(klass.get(newid, 'title'), 'Mr Friendly') - self.assertEqual(len(klass.list()), n) - if klass.do_journal: - self.assertNotEqual(klass.history(newid), []) - - def testExceptions(self): - # this tests the exceptions that should be raised - ar = self.assertRaises - - # - # class create - # - # string property - ar(TypeError, self.db.status.create, name=1) - # invalid property name - ar(KeyError, self.db.status.create, foo='foo') - # key name clash - ar(ValueError, self.db.status.create, name='unread') - # invalid link index - ar(IndexError, self.db.issue.create, title='foo', status='bar') - # invalid link value - ar(ValueError, self.db.issue.create, title='foo', status=1) - # invalid multilink type - ar(TypeError, self.db.issue.create, title='foo', status='1', - nosy='hello') - # invalid multilink index type - ar(ValueError, self.db.issue.create, title='foo', status='1', - nosy=[1]) - # invalid multilink index - ar(IndexError, self.db.issue.create, title='foo', status='1', - nosy=['10']) - - # - # key property - # - # key must be a String - ar(TypeError, self.db.file.setkey, 'fooz') - # key must exist - ar(KeyError, self.db.file.setkey, 'fubar') - - # - # class get - # - # invalid node id - ar(IndexError, self.db.issue.get, '99', 'title') - # invalid property name - ar(KeyError, self.db.status.get, '2', 'foo') - - # - # class set - # - # invalid node id - ar(IndexError, self.db.issue.set, '99', title='foo') - # invalid property name - ar(KeyError, self.db.status.set, '1', foo='foo') - # string property - ar(TypeError, self.db.status.set, '1', name=1) - # key name clash - ar(ValueError, self.db.status.set, '2', name='unread') - # set up a valid issue for me to work on - id = self.db.issue.create(title="spam", status='1') - # invalid link index - ar(IndexError, self.db.issue.set, id, title='foo', status='bar') - # invalid link value - ar(ValueError, self.db.issue.set, id, title='foo', status=1) - # invalid multilink type - ar(TypeError, self.db.issue.set, id, title='foo', status='1', - nosy='hello') - # invalid multilink index type - ar(ValueError, self.db.issue.set, id, title='foo', status='1', - nosy=[1]) - # invalid multilink index - ar(IndexError, self.db.issue.set, id, title='foo', status='1', - nosy=['10']) - # invalid number value - ar(TypeError, self.db.user.create, username='foo', age='a') - # invalid boolean value - ar(TypeError, self.db.user.create, username='foo', assignable='true') - nid = self.db.user.create(username='foo') - # invalid number value - ar(TypeError, self.db.user.set, nid, username='foo', age='a') - # invalid boolean value - ar(TypeError, self.db.user.set, nid, username='foo', assignable='true') - - def testJournals(self): - self.db.user.create(username="mary") - self.db.user.create(username="pete") - self.db.issue.create(title="spam", status='1') - self.db.commit() - - # journal entry for issue create - journal = self.db.getjournal('issue', '1') - self.assertEqual(1, len(journal)) - (nodeid, date_stamp, journaltag, action, params) = journal[0] - self.assertEqual(nodeid, '1') - self.assertEqual(journaltag, self.db.user.lookup('admin')) - self.assertEqual(action, 'create') - keys = params.keys() - keys.sort() - self.assertEqual(keys, ['assignedto', 'deadline', 'files', - 'foo', 'messages', 'nosy', 'status', 'superseder', 'title']) - self.assertEqual(None,params['deadline']) - self.assertEqual(None,params['foo']) - self.assertEqual([],params['nosy']) - self.assertEqual('1',params['status']) - self.assertEqual('spam',params['title']) - - # journal entry for link - journal = self.db.getjournal('user', '1') - self.assertEqual(1, len(journal)) - self.db.issue.set('1', assignedto='1') - self.db.commit() - journal = self.db.getjournal('user', '1') - self.assertEqual(2, len(journal)) - (nodeid, date_stamp, journaltag, action, params) = journal[1] - self.assertEqual('1', nodeid) - self.assertEqual('1', journaltag) - self.assertEqual('link', action) - self.assertEqual(('issue', '1', 'assignedto'), params) - - # journal entry for unlink - self.db.issue.set('1', assignedto='2') - self.db.commit() - journal = self.db.getjournal('user', '1') - self.assertEqual(3, len(journal)) - (nodeid, date_stamp, journaltag, action, params) = journal[2] - self.assertEqual('1', nodeid) - self.assertEqual('1', journaltag) - self.assertEqual('unlink', action) - self.assertEqual(('issue', '1', 'assignedto'), params) - - # test disabling journalling - # ... get the last entry - time.sleep(1) - entry = self.db.getjournal('issue', '1')[-1] - (x, date_stamp, x, x, x) = entry - self.db.issue.disableJournalling() - self.db.issue.set('1', title='hello world') - self.db.commit() - entry = self.db.getjournal('issue', '1')[-1] - (x, date_stamp2, x, x, x) = entry - # see if the change was journalled when it shouldn't have been - self.assertEqual(date_stamp, date_stamp2) - time.sleep(1) - self.db.issue.enableJournalling() - self.db.issue.set('1', title='hello world 2') - self.db.commit() - entry = self.db.getjournal('issue', '1')[-1] - (x, date_stamp2, x, x, x) = entry - # see if the change was journalled - self.assertNotEqual(date_stamp, date_stamp2) - - def testPack(self): - id = self.db.issue.create(title="spam", status='1') - self.db.commit() - self.db.issue.set(id, status='2') - self.db.commit() - - # sleep for at least a second, then get a date to pack at - time.sleep(1) - pack_before = date.Date('.') - - # wait another second and add one more entry - time.sleep(1) - self.db.issue.set(id, status='3') - self.db.commit() - jlen = len(self.db.getjournal('issue', id)) - - # pack - self.db.pack(pack_before) - - # we should have the create and last set entries now - self.assertEqual(jlen-1, len(self.db.getjournal('issue', id))) - - def testIDGeneration(self): - id1 = self.db.issue.create(title="spam", status='1') - id2 = self.db2.issue.create(title="eggs", status='2') - self.assertNotEqual(id1, id2) - - def testSearching(self): - self.db.file.create(content='hello', type="text/plain") - self.db.file.create(content='world', type="text/frozz", - comment='blah blah') - self.db.issue.create(files=['1', '2'], title="flebble plop") - self.db.issue.create(title="flebble frooz") - self.db.commit() - self.assertEquals(self.db.indexer.search(['hello'], self.db.issue), - {'1': {'files': ['1']}}) - self.assertEquals(self.db.indexer.search(['world'], self.db.issue), {}) - self.assertEquals(self.db.indexer.search(['frooz'], self.db.issue), - {'2': {}}) - self.assertEquals(self.db.indexer.search(['flebble'], self.db.issue), - {'2': {}, '1': {}}) - - def testReindexing(self): - self.db.issue.create(title="frooz") - self.db.commit() - self.assertEquals(self.db.indexer.search(['frooz'], self.db.issue), - {'1': {}}) - self.db.issue.set('1', title="dooble") - self.db.commit() - self.assertEquals(self.db.indexer.search(['dooble'], self.db.issue), - {'1': {}}) - self.assertEquals(self.db.indexer.search(['frooz'], self.db.issue), {}) - - def testForcedReindexing(self): - self.db.issue.create(title="flebble frooz") - self.db.commit() - self.assertEquals(self.db.indexer.search(['flebble'], self.db.issue), - {'1': {}}) - self.db.indexer.quiet = 1 - self.db.indexer.force_reindex() - self.db.post_init() - self.db.indexer.quiet = 9 - self.assertEquals(self.db.indexer.search(['flebble'], self.db.issue), - {'1': {}}) - - # - # searching tests follow - # - def testFind(self): - self.db.user.create(username='test') - ids = [] - ids.append(self.db.issue.create(status="1", nosy=['1'])) - oddid = self.db.issue.create(status="2", nosy=['2']) - ids.append(self.db.issue.create(status="1", nosy=['1','2'])) - self.db.issue.create(status="3", nosy=['1']) - ids.sort() - - # should match first and third - got = self.db.issue.find(status='1') - got.sort() - self.assertEqual(got, ids) - - # none - self.assertEqual(self.db.issue.find(status='4'), []) - - # should match first three - got = self.db.issue.find(status='1', nosy='2') - got.sort() - ids.append(oddid) - ids.sort() - self.assertEqual(got, ids) - - # none - self.assertEqual(self.db.issue.find(status='4', nosy='3'), []) - - def testStringFind(self): - ids = [] - ids.append(self.db.issue.create(title="spam")) - self.db.issue.create(title="not spam") - ids.append(self.db.issue.create(title="spam")) - ids.sort() - got = self.db.issue.stringFind(title='spam') - got.sort() - self.assertEqual(got, ids) - self.assertEqual(self.db.issue.stringFind(title='fubar'), []) - - def filteringSetup(self): - for user in ( - {'username': 'bleep'}, - {'username': 'blop'}, - {'username': 'blorp'}): - self.db.user.create(**user) - iss = self.db.issue - for issue in ( - {'title': 'issue one', 'status': '2'}, - {'title': 'issue two', 'status': '1'}, - {'title': 'issue three', 'status': '1', 'nosy': ['1','2']}): - self.db.issue.create(**issue) - self.db.commit() - return self.assertEqual, self.db.issue.filter - - def testFilteringString(self): - ae, filt = self.filteringSetup() - ae(filt(None, {'title': 'issue one'}, ('+','id'), (None,None)), ['1']) - ae(filt(None, {'title': 'issue'}, ('+','id'), (None,None)), - ['1','2','3']) - - def testFilteringLink(self): - ae, filt = self.filteringSetup() - ae(filt(None, {'status': '1'}, ('+','id'), (None,None)), ['2','3']) - - def testFilteringMultilink(self): - ae, filt = self.filteringSetup() - ae(filt(None, {'nosy': '2'}, ('+','id'), (None,None)), ['3']) - - def testFilteringMany(self): - ae, filt = self.filteringSetup() - ae(filt(None, {'nosy': '2', 'status': '1'}, ('+','id'), (None,None)), - ['3']) - -class anydbmReadOnlyDBTestCase(MyTestCase): - def setUp(self): - from roundup.backends import anydbm - # remove previous test, ignore errors - if os.path.exists(config.DATABASE): - shutil.rmtree(config.DATABASE) - os.makedirs(config.DATABASE + '/files') - db = anydbm.Database(config, 'admin') - setupSchema(db, 1, anydbm) - self.db = anydbm.Database(config) - setupSchema(self.db, 0, anydbm) - self.db2 = anydbm.Database(config, 'admin') - setupSchema(self.db2, 0, anydbm) - - def testExceptions(self): - # this tests the exceptions that should be raised - ar = self.assertRaises - - # this tests the exceptions that should be raised - ar(DatabaseError, self.db.status.create, name="foo") - ar(DatabaseError, self.db.status.set, '1', name="foo") - ar(DatabaseError, self.db.status.retire, '1') - - -class bsddbDBTestCase(anydbmDBTestCase): - def setUp(self): - from roundup.backends import bsddb - # remove previous test, ignore errors - if os.path.exists(config.DATABASE): - shutil.rmtree(config.DATABASE) - os.makedirs(config.DATABASE + '/files') - self.db = bsddb.Database(config, 'admin') - setupSchema(self.db, 1, bsddb) - self.db2 = bsddb.Database(config, 'admin') - setupSchema(self.db2, 0, bsddb) - -class bsddbReadOnlyDBTestCase(anydbmReadOnlyDBTestCase): - def setUp(self): - from roundup.backends import bsddb - # remove previous test, ignore errors - if os.path.exists(config.DATABASE): - shutil.rmtree(config.DATABASE) - os.makedirs(config.DATABASE + '/files') - db = bsddb.Database(config, 'admin') - setupSchema(db, 1, bsddb) - self.db = bsddb.Database(config) - setupSchema(self.db, 0, bsddb) - self.db2 = bsddb.Database(config, 'admin') - setupSchema(self.db2, 0, bsddb) - - -class bsddb3DBTestCase(anydbmDBTestCase): - def setUp(self): - from roundup.backends import bsddb3 - # remove previous test, ignore errors - if os.path.exists(config.DATABASE): - shutil.rmtree(config.DATABASE) - os.makedirs(config.DATABASE + '/files') - self.db = bsddb3.Database(config, 'admin') - setupSchema(self.db, 1, bsddb3) - self.db2 = bsddb3.Database(config, 'admin') - setupSchema(self.db2, 0, bsddb3) - -class bsddb3ReadOnlyDBTestCase(anydbmReadOnlyDBTestCase): - def setUp(self): - from roundup.backends import bsddb3 - # remove previous test, ignore errors - if os.path.exists(config.DATABASE): - shutil.rmtree(config.DATABASE) - os.makedirs(config.DATABASE + '/files') - db = bsddb3.Database(config, 'admin') - setupSchema(db, 1, bsddb3) - self.db = bsddb3.Database(config) - setupSchema(self.db, 0, bsddb3) - self.db2 = bsddb3.Database(config, 'admin') - setupSchema(self.db2, 0, bsddb3) - - -class gadflyDBTestCase(anydbmDBTestCase): - ''' Gadfly doesn't support multiple connections to the one local - database - ''' - def setUp(self): - from roundup.backends import gadfly - # remove previous test, ignore errors - if os.path.exists(config.DATABASE): - shutil.rmtree(config.DATABASE) - config.GADFLY_DATABASE = ('test', config.DATABASE) - os.makedirs(config.DATABASE + '/files') - self.db = gadfly.Database(config, 'admin') - setupSchema(self.db, 1, gadfly) - - def testIDGeneration(self): - id1 = self.db.issue.create(title="spam", status='1') - id2 = self.db.issue.create(title="eggs", status='2') - self.assertNotEqual(id1, id2) - - def testFilteringString(self): - ae, filt = self.filteringSetup() - ae(filt(None, {'title': 'issue one'}, ('+','id'), (None,None)), ['1']) - # XXX gadfly can't do substring LIKE searches - #ae(filt(None, {'title': 'issue'}, ('+','id'), (None,None)), - # ['1','2','3']) - -class gadflyReadOnlyDBTestCase(anydbmReadOnlyDBTestCase): - def setUp(self): - from roundup.backends import gadfly - # remove previous test, ignore errors - if os.path.exists(config.DATABASE): - shutil.rmtree(config.DATABASE) - config.GADFLY_DATABASE = ('test', config.DATABASE) - os.makedirs(config.DATABASE + '/files') - db = gadfly.Database(config, 'admin') - setupSchema(db, 1, gadfly) - self.db = gadfly.Database(config) - setupSchema(self.db, 0, gadfly) - - -class sqliteDBTestCase(anydbmDBTestCase): - def setUp(self): - from roundup.backends import sqlite - # remove previous test, ignore errors - if os.path.exists(config.DATABASE): - shutil.rmtree(config.DATABASE) - os.makedirs(config.DATABASE + '/files') - self.db = sqlite.Database(config, 'admin') - setupSchema(self.db, 1, sqlite) - - def testIDGeneration(self): - pass - -class sqliteReadOnlyDBTestCase(anydbmReadOnlyDBTestCase): - def setUp(self): - from roundup.backends import sqlite - # remove previous test, ignore errors - if os.path.exists(config.DATABASE): - shutil.rmtree(config.DATABASE) - os.makedirs(config.DATABASE + '/files') - db = sqlite.Database(config, 'admin') - setupSchema(db, 1, sqlite) - self.db = sqlite.Database(config) - setupSchema(self.db, 0, sqlite) - - -class metakitDBTestCase(anydbmDBTestCase): - def setUp(self): - from roundup.backends import metakit - import weakref - metakit._instances = weakref.WeakValueDictionary() - # remove previous test, ignore errors - if os.path.exists(config.DATABASE): - shutil.rmtree(config.DATABASE) - os.makedirs(config.DATABASE + '/files') - self.db = metakit.Database(config, 'admin') - setupSchema(self.db, 1, metakit) - - def testIDGeneration(self): - id1 = self.db.issue.create(title="spam", status='1') - id2 = self.db.issue.create(title="eggs", status='2') - self.assertNotEqual(id1, id2) - - def testTransactions(self): - # remember the number of items we started - num_issues = len(self.db.issue.list()) - self.db.issue.create(title="don't commit me!", status='1') - self.assertNotEqual(num_issues, len(self.db.issue.list())) - self.db.rollback() - self.assertEqual(num_issues, len(self.db.issue.list())) - self.db.issue.create(title="please commit me!", status='1') - self.assertNotEqual(num_issues, len(self.db.issue.list())) - self.db.commit() - self.assertNotEqual(num_issues, len(self.db.issue.list())) - self.db.rollback() - self.assertNotEqual(num_issues, len(self.db.issue.list())) - self.db.file.create(name="test", type="text/plain", content="hi") - self.db.rollback() - for i in range(10): - self.db.file.create(name="test", type="text/plain", - content="hi %d"%(i)) - self.db.commit() - # TODO: would be good to be able to ensure the file is not on disk after - # a rollback... - self.assertNotEqual(num_files, num_files2) - self.db.file.create(name="test", type="text/plain", content="hi") - self.db.rollback() - -class metakitReadOnlyDBTestCase(anydbmReadOnlyDBTestCase): - def setUp(self): - from roundup.backends import metakit - import weakref - metakit._instances = weakref.WeakValueDictionary() - # remove previous test, ignore errors - if os.path.exists(config.DATABASE): - shutil.rmtree(config.DATABASE) - os.makedirs(config.DATABASE + '/files') - db = metakit.Database(config, 'admin') - setupSchema(db, 1, metakit) - self.db = metakit.Database(config) - setupSchema(self.db, 0, metakit) - -def suite(): - l = [ - unittest.makeSuite(anydbmDBTestCase, 'test'), - unittest.makeSuite(anydbmReadOnlyDBTestCase, 'test') - ] -# return unittest.TestSuite(l) - - from roundup import backends - if hasattr(backends, 'gadfly'): - l.append(unittest.makeSuite(gadflyDBTestCase, 'test')) - l.append(unittest.makeSuite(gadflyReadOnlyDBTestCase, 'test')) - - if hasattr(backends, 'sqlite'): - l.append(unittest.makeSuite(sqliteDBTestCase, 'test')) - l.append(unittest.makeSuite(sqliteReadOnlyDBTestCase, 'test')) - - if hasattr(backends, 'bsddb'): - l.append(unittest.makeSuite(bsddbDBTestCase, 'test')) - l.append(unittest.makeSuite(bsddbReadOnlyDBTestCase, 'test')) - - if hasattr(backends, 'bsddb3'): - l.append(unittest.makeSuite(bsddb3DBTestCase, 'test')) - l.append(unittest.makeSuite(bsddb3ReadOnlyDBTestCase, 'test')) - - if hasattr(backends, 'metakit'): - l.append(unittest.makeSuite(metakitDBTestCase, 'test')) - l.append(unittest.makeSuite(metakitReadOnlyDBTestCase, 'test')) - - return unittest.TestSuite(l) - -# vim: set filetype=python ts=4 sw=4 et si
--- a/test/test_locking.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,51 +0,0 @@ -# Copyright (c) 2002 ekit.com Inc (http://www.ekit-inc.com/) -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in -# all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -# SOFTWARE. - -# $Id: test_locking.py,v 1.2 2002-09-10 00:19:54 richard Exp $ - -import os, unittest, tempfile - -from roundup.backends.locking import acquire_lock, release_lock - -class LockingTest(unittest.TestCase): - def setUp(self): - self.path = tempfile.mktemp() - open(self.path, 'w').write('hi\n') - - def test_basics(self): - f = acquire_lock(self.path) - try: - acquire_lock(self.path, block=0) - except: - pass - else: - raise AssertionError, 'no exception' - release_lock(f) - f = acquire_lock(self.path) - release_lock(f) - - def tearDown(self): - os.remove(self.path) - -def suite(): - return unittest.makeSuite(LockingTest) - - -# vim: set filetype=python ts=4 sw=4 et si
--- a/test/test_mailgw.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,827 +0,0 @@ -# -# Copyright (c) 2001 Richard Jones, richard@bofh.asn.au. -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# This module is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. -# -# $Id: test_mailgw.py,v 1.32 2002-09-26 03:04:24 richard Exp $ - -import unittest, cStringIO, tempfile, os, shutil, errno, imp, sys, difflib - -# Note: Should parse emails according to RFC2822 instead of performing a -# literal string comparision. Parsing the messages allows the tests to work for -# any legal serialization of an email. -#try : -# import email -#except ImportError : -# import rfc822 as email - -from roundup.mailgw import MailGW, Unauthorized -from roundup import init, instance - -# TODO: make this output only enough equal lines for context, not all of -# them -class DiffHelper: - def compareStrings(self, s2, s1): - '''Note the reversal of s2 and s1 - difflib.SequenceMatcher wants - the first to be the "original" but in the calls in this file, - the second arg is the original. Ho hum. - ''' - if s1 == s2: - return - - # under python2.[12] we allow a difference of one trailing empty line. - if sys.version_info[0:2] == (2,1): - if s1+'\n' == s2: - return - if sys.version_info[0:2] == (2,2): - if s1 == s2+'\n': - return - - l1=s1.split('\n') - l2=s2.split('\n') - s = difflib.SequenceMatcher(None, l1, l2) - res = ['Generated message not correct (diff follows):'] - for value, s1s, s1e, s2s, s2e in s.get_opcodes(): - if value == 'equal': - for i in range(s1s, s1e): - res.append(' %s'%l1[i]) - elif value == 'delete': - for i in range(s1s, s1e): - res.append('- %s'%l1[i]) - elif value == 'insert': - for i in range(s2s, s2e): - res.append('+ %s'%l2[i]) - elif value == 'replace': - for i, j in zip(range(s1s, s1e), range(s2s, s2e)): - res.append('- %s'%l1[i]) - res.append('+ %s'%l2[j]) - - raise AssertionError, '\n'.join(res) - -class MailgwTestCase(unittest.TestCase, DiffHelper): - count = 0 - schema = 'classic' - def setUp(self): - MailgwTestCase.count = MailgwTestCase.count + 1 - self.dirname = '_test_mailgw_%s'%self.count - try: - shutil.rmtree(self.dirname) - except OSError, error: - if error.errno not in (errno.ENOENT, errno.ESRCH): raise - # create the instance - init.install(self.dirname, 'classic', 'anydbm') - init.initialise(self.dirname, 'sekrit') - # check we can load the package - self.instance = instance.open(self.dirname) - # and open the database - self.db = self.instance.open('admin') - self.db.user.create(username='Chef', address='chef@bork.bork.bork', - roles='User') - self.db.user.create(username='richard', address='richard@test', - roles='User') - self.db.user.create(username='mary', address='mary@test', - roles='User') - self.db.user.create(username='john', address='john@test', - alternate_addresses='jondoe@test\njohn.doe@test', roles='User') - - def tearDown(self): - if os.path.exists(os.environ['SENDMAILDEBUG']): - os.remove(os.environ['SENDMAILDEBUG']) - self.db.close() - try: - shutil.rmtree(self.dirname) - except OSError, error: - if error.errno not in (errno.ENOENT, errno.ESRCH): raise - - def doNewIssue(self): - message = cStringIO.StringIO('''Content-Type: text/plain; - charset="iso-8859-1" -From: Chef <chef@bork.bork.bork> -To: issue_tracker@your.tracker.email.domain.example -Cc: richard@test -Message-Id: <dummy_test_message_id> -Subject: [issue] Testing... - -This is a test submission of a new issue. -''') - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - nodeid = handler.main(message) - if os.path.exists(os.environ['SENDMAILDEBUG']): - error = open(os.environ['SENDMAILDEBUG']).read() - self.assertEqual('no error', error) - l = self.db.issue.get(nodeid, 'nosy') - l.sort() - self.assertEqual(l, ['3', '4']) - - def testNewIssue(self): - self.doNewIssue() - - def testNewIssueNosy(self): - self.instance.config.ADD_AUTHOR_TO_NOSY = 'yes' - message = cStringIO.StringIO('''Content-Type: text/plain; - charset="iso-8859-1" -From: Chef <chef@bork.bork.bork> -To: issue_tracker@your.tracker.email.domain.example -Cc: richard@test -Message-Id: <dummy_test_message_id> -Subject: [issue] Testing... - -This is a test submission of a new issue. -''') - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - nodeid = handler.main(message) - if os.path.exists(os.environ['SENDMAILDEBUG']): - error = open(os.environ['SENDMAILDEBUG']).read() - self.assertEqual('no error', error) - l = self.db.issue.get(nodeid, 'nosy') - l.sort() - self.assertEqual(l, ['3', '4']) - - def testAlternateAddress(self): - message = cStringIO.StringIO('''Content-Type: text/plain; - charset="iso-8859-1" -From: John Doe <john.doe@test> -To: issue_tracker@your.tracker.email.domain.example -Message-Id: <dummy_test_message_id> -Subject: [issue] Testing... - -This is a test submission of a new issue. -''') - userlist = self.db.user.list() - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - handler.main(message) - if os.path.exists(os.environ['SENDMAILDEBUG']): - error = open(os.environ['SENDMAILDEBUG']).read() - self.assertEqual('no error', error) - self.assertEqual(userlist, self.db.user.list(), - "user created when it shouldn't have been") - - def testNewIssueNoClass(self): - message = cStringIO.StringIO('''Content-Type: text/plain; - charset="iso-8859-1" -From: Chef <chef@bork.bork.bork> -To: issue_tracker@your.tracker.email.domain.example -Cc: richard@test -Message-Id: <dummy_test_message_id> -Subject: Testing... - -This is a test submission of a new issue. -''') - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - handler.main(message) - if os.path.exists(os.environ['SENDMAILDEBUG']): - error = open(os.environ['SENDMAILDEBUG']).read() - self.assertEqual('no error', error) - - def testNewIssueAuthMsg(self): - message = cStringIO.StringIO('''Content-Type: text/plain; - charset="iso-8859-1" -From: Chef <chef@bork.bork.bork> -To: issue_tracker@your.tracker.email.domain.example -Message-Id: <dummy_test_message_id> -Subject: [issue] Testing... [nosy=mary; assignedto=richard] - -This is a test submission of a new issue. -''') - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - # TODO: fix the damn config - this is apalling - self.db.config.MESSAGES_TO_AUTHOR = 'yes' - handler.main(message) - - self.compareStrings(open(os.environ['SENDMAILDEBUG']).read(), -'''FROM: roundup-admin@your.tracker.email.domain.example -TO: chef@bork.bork.bork, mary@test, richard@test -Content-Type: text/plain -Subject: [issue1] Testing... -To: chef@bork.bork.bork, mary@test, richard@test -From: "Chef" <issue_tracker@your.tracker.email.domain.example> -Reply-To: "Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -MIME-Version: 1.0 -Message-Id: <dummy_test_message_id> -X-Roundup-Name: Roundup issue tracker -Content-Transfer-Encoding: quoted-printable - - -New submission from Chef <chef@bork.bork.bork>: - -This is a test submission of a new issue. - - ----------- -assignedto: richard -messages: 1 -nosy: Chef, mary, richard -status: unread -title: Testing... -_________________________________________________________________________ -"Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -http://your.tracker.url.example/issue1 -_________________________________________________________________________ -''') - - # BUG - # def testMultipart(self): - # '''With more than one part''' - # see MultipartEnc tests: but if there is more than one part - # we return a multipart/mixed and the boundary contains - # the ip address of the test machine. - - # BUG should test some binary attamchent too. - - def testSimpleFollowup(self): - self.doNewIssue() - message = cStringIO.StringIO('''Content-Type: text/plain; - charset="iso-8859-1" -From: mary <mary@test> -To: issue_tracker@your.tracker.email.domain.example -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -Subject: [issue1] Testing... - -This is a second followup -''') - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - handler.main(message) - self.compareStrings(open(os.environ['SENDMAILDEBUG']).read(), -'''FROM: roundup-admin@your.tracker.email.domain.example -TO: chef@bork.bork.bork, richard@test -Content-Type: text/plain -Subject: [issue1] Testing... -To: chef@bork.bork.bork, richard@test -From: "mary" <issue_tracker@your.tracker.email.domain.example> -Reply-To: "Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -MIME-Version: 1.0 -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -X-Roundup-Name: Roundup issue tracker -Content-Transfer-Encoding: quoted-printable - - -mary <mary@test> added the comment: - -This is a second followup - - ----------- -status: unread -> chatting -_________________________________________________________________________ -"Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -http://your.tracker.url.example/issue1 -_________________________________________________________________________ -''') - - def testFollowup(self): - self.doNewIssue() - - message = cStringIO.StringIO('''Content-Type: text/plain; - charset="iso-8859-1" -From: richard <richard@test> -To: issue_tracker@your.tracker.email.domain.example -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -Subject: [issue1] Testing... [assignedto=mary; nosy=+john] - -This is a followup -''') - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - handler.main(message) - l = self.db.issue.get('1', 'nosy') - l.sort() - self.assertEqual(l, ['3', '4', '5', '6']) - - self.compareStrings(open(os.environ['SENDMAILDEBUG']).read(), -'''FROM: roundup-admin@your.tracker.email.domain.example -TO: chef@bork.bork.bork, john@test, mary@test -Content-Type: text/plain -Subject: [issue1] Testing... -To: chef@bork.bork.bork, john@test, mary@test -From: "richard" <issue_tracker@your.tracker.email.domain.example> -Reply-To: "Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -MIME-Version: 1.0 -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -X-Roundup-Name: Roundup issue tracker -Content-Transfer-Encoding: quoted-printable - - -richard <richard@test> added the comment: - -This is a followup - - ----------- -assignedto: -> mary -nosy: +john, mary -status: unread -> chatting -_________________________________________________________________________ -"Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -http://your.tracker.url.example/issue1 -_________________________________________________________________________ -''') - - def testFollowupTitleMatch(self): - self.doNewIssue() - message = cStringIO.StringIO('''Content-Type: text/plain; - charset="iso-8859-1" -From: richard <richard@test> -To: issue_tracker@your.tracker.email.domain.example -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -Subject: Re: Testing... [assignedto=mary; nosy=+john] - -This is a followup -''') - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - handler.main(message) - - self.compareStrings(open(os.environ['SENDMAILDEBUG']).read(), -'''FROM: roundup-admin@your.tracker.email.domain.example -TO: chef@bork.bork.bork, john@test, mary@test -Content-Type: text/plain -Subject: [issue1] Testing... -To: chef@bork.bork.bork, john@test, mary@test -From: "richard" <issue_tracker@your.tracker.email.domain.example> -Reply-To: "Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -MIME-Version: 1.0 -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -X-Roundup-Name: Roundup issue tracker -Content-Transfer-Encoding: quoted-printable - - -richard <richard@test> added the comment: - -This is a followup - - ----------- -assignedto: -> mary -nosy: +john, mary -status: unread -> chatting -_________________________________________________________________________ -"Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -http://your.tracker.url.example/issue1 -_________________________________________________________________________ -''') - - def testFollowupNosyAuthor(self): - self.doNewIssue() - self.db.config.ADD_AUTHOR_TO_NOSY = 'yes' - message = cStringIO.StringIO('''Content-Type: text/plain; - charset="iso-8859-1" -From: john@test -To: issue_tracker@your.tracker.email.domain.example -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -Subject: [issue1] Testing... - -This is a followup -''') - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - handler.main(message) - - self.compareStrings(open(os.environ['SENDMAILDEBUG']).read(), -'''FROM: roundup-admin@your.tracker.email.domain.example -TO: chef@bork.bork.bork, richard@test -Content-Type: text/plain -Subject: [issue1] Testing... -To: chef@bork.bork.bork, richard@test -From: "john" <issue_tracker@your.tracker.email.domain.example> -Reply-To: "Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -MIME-Version: 1.0 -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -X-Roundup-Name: Roundup issue tracker -Content-Transfer-Encoding: quoted-printable - - -john <john@test> added the comment: - -This is a followup - - ----------- -nosy: +john -status: unread -> chatting -_________________________________________________________________________ -"Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -http://your.tracker.url.example/issue1 -_________________________________________________________________________ - -''') - - def testFollowupNosyRecipients(self): - self.doNewIssue() - self.db.config.ADD_RECIPIENTS_TO_NOSY = 'yes' - message = cStringIO.StringIO('''Content-Type: text/plain; - charset="iso-8859-1" -From: richard@test -To: issue_tracker@your.tracker.email.domain.example -Cc: john@test -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -Subject: [issue1] Testing... - -This is a followup -''') - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - handler.main(message) - - self.compareStrings(open(os.environ['SENDMAILDEBUG']).read(), -'''FROM: roundup-admin@your.tracker.email.domain.example -TO: chef@bork.bork.bork -Content-Type: text/plain -Subject: [issue1] Testing... -To: chef@bork.bork.bork -From: "richard" <issue_tracker@your.tracker.email.domain.example> -Reply-To: "Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -MIME-Version: 1.0 -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -X-Roundup-Name: Roundup issue tracker -Content-Transfer-Encoding: quoted-printable - - -richard <richard@test> added the comment: - -This is a followup - - ----------- -nosy: +john -status: unread -> chatting -_________________________________________________________________________ -"Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -http://your.tracker.url.example/issue1 -_________________________________________________________________________ - -''') - - def testFollowupNosyAuthorAndCopy(self): - self.doNewIssue() - self.db.config.ADD_AUTHOR_TO_NOSY = 'yes' - self.db.config.MESSAGES_TO_AUTHOR = 'yes' - message = cStringIO.StringIO('''Content-Type: text/plain; - charset="iso-8859-1" -From: john@test -To: issue_tracker@your.tracker.email.domain.example -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -Subject: [issue1] Testing... - -This is a followup -''') - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - handler.main(message) - - self.compareStrings(open(os.environ['SENDMAILDEBUG']).read(), -'''FROM: roundup-admin@your.tracker.email.domain.example -TO: chef@bork.bork.bork, john@test, richard@test -Content-Type: text/plain -Subject: [issue1] Testing... -To: chef@bork.bork.bork, john@test, richard@test -From: "john" <issue_tracker@your.tracker.email.domain.example> -Reply-To: "Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -MIME-Version: 1.0 -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -X-Roundup-Name: Roundup issue tracker -Content-Transfer-Encoding: quoted-printable - - -john <john@test> added the comment: - -This is a followup - - ----------- -nosy: +john -status: unread -> chatting -_________________________________________________________________________ -"Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -http://your.tracker.url.example/issue1 -_________________________________________________________________________ - -''') - - def testFollowupNoNosyAuthor(self): - self.doNewIssue() - self.instance.config.ADD_AUTHOR_TO_NOSY = 'no' - message = cStringIO.StringIO('''Content-Type: text/plain; - charset="iso-8859-1" -From: john@test -To: issue_tracker@your.tracker.email.domain.example -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -Subject: [issue1] Testing... - -This is a followup -''') - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - handler.main(message) - - self.compareStrings(open(os.environ['SENDMAILDEBUG']).read(), -'''FROM: roundup-admin@your.tracker.email.domain.example -TO: chef@bork.bork.bork, richard@test -Content-Type: text/plain -Subject: [issue1] Testing... -To: chef@bork.bork.bork, richard@test -From: "john" <issue_tracker@your.tracker.email.domain.example> -Reply-To: "Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -MIME-Version: 1.0 -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -X-Roundup-Name: Roundup issue tracker -Content-Transfer-Encoding: quoted-printable - - -john <john@test> added the comment: - -This is a followup - - ----------- -status: unread -> chatting -_________________________________________________________________________ -"Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -http://your.tracker.url.example/issue1 -_________________________________________________________________________ - -''') - - def testFollowupNoNosyRecipients(self): - self.doNewIssue() - self.instance.config.ADD_RECIPIENTS_TO_NOSY = 'no' - message = cStringIO.StringIO('''Content-Type: text/plain; - charset="iso-8859-1" -From: richard@test -To: issue_tracker@your.tracker.email.domain.example -Cc: john@test -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -Subject: [issue1] Testing... - -This is a followup -''') - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - handler.main(message) - - self.compareStrings(open(os.environ['SENDMAILDEBUG']).read(), -'''FROM: roundup-admin@your.tracker.email.domain.example -TO: chef@bork.bork.bork -Content-Type: text/plain -Subject: [issue1] Testing... -To: chef@bork.bork.bork -From: "richard" <issue_tracker@your.tracker.email.domain.example> -Reply-To: "Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -MIME-Version: 1.0 -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -X-Roundup-Name: Roundup issue tracker -Content-Transfer-Encoding: quoted-printable - - -richard <richard@test> added the comment: - -This is a followup - - ----------- -status: unread -> chatting -_________________________________________________________________________ -"Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -http://your.tracker.url.example/issue1 -_________________________________________________________________________ - -''') - - def testNosyRemove(self): - self.doNewIssue() - - message = cStringIO.StringIO('''Content-Type: text/plain; - charset="iso-8859-1" -From: richard <richard@test> -To: issue_tracker@your.tracker.email.domain.example -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -Subject: [issue1] Testing... [nosy=-richard] - -''') - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - handler.main(message) - l = self.db.issue.get('1', 'nosy') - l.sort() - self.assertEqual(l, ['3']) - - # NO NOSY MESSAGE SHOULD BE SENT! - self.assert_(not os.path.exists(os.environ['SENDMAILDEBUG'])) - - def testNewUserAuthor(self): - # first without the permission - # heh... just ignore the API for a second ;) - self.db.security.role['Anonymous'].permissions=[] - anonid = self.db.user.lookup('anonymous') - self.db.user.set(anonid, roles='Anonymous') - - self.db.security.hasPermission('Email Registration', anonid) - l = self.db.user.list() - l.sort() - s = '''Content-Type: text/plain; - charset="iso-8859-1" -From: fubar <fubar@bork.bork.bork> -To: issue_tracker@your.tracker.email.domain.example -Message-Id: <dummy_test_message_id> -Subject: [issue] Testing... - -This is a test submission of a new issue. -''' - message = cStringIO.StringIO(s) - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - self.assertRaises(Unauthorized, handler.main, message) - m = self.db.user.list() - m.sort() - self.assertEqual(l, m) - - # now with the permission - p = self.db.security.getPermission('Email Registration') - self.db.security.role['Anonymous'].permissions=[p] - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - message = cStringIO.StringIO(s) - handler.main(message) - m = self.db.user.list() - m.sort() - self.assertNotEqual(l, m) - - def testEnc01(self): - self.doNewIssue() - message = cStringIO.StringIO('''Content-Type: text/plain; - charset="iso-8859-1" -From: mary <mary@test> -To: issue_tracker@your.tracker.email.domain.example -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -Subject: [issue1] Testing... -Content-Type: text/plain; - charset="iso-8859-1" -Content-Transfer-Encoding: quoted-printable - -A message with encoding (encoded oe =F6) - -''') - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - handler.main(message) - self.compareStrings(open(os.environ['SENDMAILDEBUG']).read(), -'''FROM: roundup-admin@your.tracker.email.domain.example -TO: chef@bork.bork.bork, richard@test -Content-Type: text/plain -Subject: [issue1] Testing... -To: chef@bork.bork.bork, richard@test -From: "mary" <issue_tracker@your.tracker.email.domain.example> -Reply-To: "Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -MIME-Version: 1.0 -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -X-Roundup-Name: Roundup issue tracker -Content-Transfer-Encoding: quoted-printable - - -mary <mary@test> added the comment: - -A message with encoding (encoded oe =F6) - ----------- -status: unread -> chatting -_________________________________________________________________________ -"Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -http://your.tracker.url.example/issue1 -_________________________________________________________________________ -''') - - - def testMultipartEnc01(self): - self.doNewIssue() - message = cStringIO.StringIO('''Content-Type: text/plain; - charset="iso-8859-1" -From: mary <mary@test> -To: issue_tracker@your.tracker.email.domain.example -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -Subject: [issue1] Testing... -Content-Type: multipart/mixed; - boundary="----_=_NextPart_000_01" - -This message is in MIME format. Since your mail reader does not understand -this format, some or all of this message may not be legible. - -------_=_NextPart_000_01 -Content-Type: text/plain; - charset="iso-8859-1" -Content-Transfer-Encoding: quoted-printable - -A message with first part encoded (encoded oe =F6) - -''') - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - handler.main(message) - self.compareStrings(open(os.environ['SENDMAILDEBUG']).read(), -'''FROM: roundup-admin@your.tracker.email.domain.example -TO: chef@bork.bork.bork, richard@test -Content-Type: text/plain -Subject: [issue1] Testing... -To: chef@bork.bork.bork, richard@test -From: "mary" <issue_tracker@your.tracker.email.domain.example> -Reply-To: "Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -MIME-Version: 1.0 -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -X-Roundup-Name: Roundup issue tracker -Content-Transfer-Encoding: quoted-printable - - -mary <mary@test> added the comment: - -A message with first part encoded (encoded oe =F6) - ----------- -status: unread -> chatting -_________________________________________________________________________ -"Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -http://your.tracker.url.example/issue1 -_________________________________________________________________________ -''') - - def testFollowupStupidQuoting(self): - self.doNewIssue() - - message = cStringIO.StringIO('''Content-Type: text/plain; - charset="iso-8859-1" -From: richard <richard@test> -To: issue_tracker@your.tracker.email.domain.example -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -Subject: Re: "[issue1] Testing... " - -This is a followup -''') - handler = self.instance.MailGW(self.instance, self.db) - handler.trapExceptions = 0 - handler.main(message) - - self.compareStrings(open(os.environ['SENDMAILDEBUG']).read(), -'''FROM: roundup-admin@your.tracker.email.domain.example -TO: chef@bork.bork.bork -Content-Type: text/plain -Subject: [issue1] Testing... -To: chef@bork.bork.bork -From: "richard" <issue_tracker@your.tracker.email.domain.example> -Reply-To: "Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -MIME-Version: 1.0 -Message-Id: <followup_dummy_id> -In-Reply-To: <dummy_test_message_id> -X-Roundup-Name: Roundup issue tracker -Content-Transfer-Encoding: quoted-printable - - -richard <richard@test> added the comment: - -This is a followup - - ----------- -status: unread -> chatting -_________________________________________________________________________ -"Roundup issue tracker" <issue_tracker@your.tracker.email.domain.example> -http://your.tracker.url.example/issue1 -_________________________________________________________________________ -''') - -def suite(): - l = [unittest.makeSuite(MailgwTestCase), - ] - return unittest.TestSuite(l) - - -# vim: set filetype=python ts=4 sw=4 et si
--- a/test/test_mailsplit.py Mon Sep 30 01:17:10 2002 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,215 +0,0 @@ -# -# Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) -# This module is free software, and you may redistribute it and/or modify -# under the same terms as Python, so long as this copyright message and -# disclaimer are retained in their original form. -# -# IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR -# DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING -# OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE -# POSSIBILITY OF SUCH DAMAGE. -# -# BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, -# BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS -# FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS" -# BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE, -# SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. -# -# $Id: test_mailsplit.py,v 1.11 2002-09-10 00:19:55 richard Exp $ - -import unittest, cStringIO - -from roundup.mailgw import parseContent - -class MailsplitTestCase(unittest.TestCase): - def testPreComment(self): - s = ''' -blah blah blah blah... blah blah? blah blah blah blah blah. blah blah blah -blah blah blah blah blah blah blah blah blah blah blah! - -issue_tracker@foo.com wrote: -> blah blah blah blahblah blahblah blahblah blah blah blah blah blah blah -> blah blah blah blah blah blah blah blah blah? blah blah blah blah blah -> blah blah blah blah blah blah blah... blah blah blah blah. blah blah -> blah blah blah blah? blah blah blah blah blah blah! blah blah! -> -> ------- -> nosy: userfoo, userken -> _________________________________________________ -> Roundup issue tracker -> issue_tracker@foo.com -> http://foo.com/cgi-bin/roundup.cgi/issue_tracker/ - --- -blah blah blah signature -userfoo@foo.com -''' - summary, content = parseContent(s, 0, 0) - self.assertEqual(summary, 'blah blah blah blah... blah blah? blah blah blah blah blah. blah blah blah') - self.assertEqual(content, 'blah blah blah blah... blah blah? blah blah blah blah blah. blah blah blah\nblah blah blah blah blah blah blah blah blah blah blah!') - - - def testPostComment(self): - s = ''' -issue_tracker@foo.com wrote: -> blah blah blah blahblah blahblah blahblah blah blah blah blah blah -> blah -> blah blah blah blah blah blah blah blah blah? blah blah blah blah -> blah -> blah blah blah blah blah blah blah... blah blah blah blah. blah -> blah -> blah blah blah blah? blah blah blah blah blah blah! blah blah! -> -> ------- -> nosy: userfoo, userken -> _________________________________________________ -> Roundup issue tracker -> issue_tracker@foo.com -> http://foo.com/cgi-bin/roundup.cgi/issue_tracker/ - -blah blah blah blah... blah blah? blah blah blah blah blah. blah blah blah -blah blah blah blah blah blah blah blah blah blah blah! - --- -blah blah blah signature -userfoo@foo.com -''' - summary, content = parseContent(s, 0, 0) - self.assertEqual(summary, 'blah blah blah blah... blah blah? blah blah blah blah blah. blah blah blah') - self.assertEqual(content, 'blah blah blah blah... blah blah? blah blah blah blah blah. blah blah blah\nblah blah blah blah blah blah blah blah blah blah blah!') - - - def testKeepCitation(self): - s = ''' -blah blah blah blah... blah blah? blah blah blah blah blah. blah blah blah -blah blah blah blah blah blah blah blah blah blah blah! - -issue_tracker@foo.com wrote: -> blah blah blah blahblah blahblah blahblah blah blah blah blah blah blah -> blah blah blah blah blah blah blah blah blah? blah blah blah blah blah -> blah blah blah blah blah blah blah... blah blah blah blah. blah blah -> blah blah blah blah? blah blah blah blah blah blah! blah blah! -> -> ------- -> nosy: userfoo, userken -> _________________________________________________ -> Roundup issue tracker -> issue_tracker@foo.com -> http://foo.com/cgi-bin/roundup.cgi/issue_tracker/ - --- -blah blah blah signature -userfoo@foo.com -''' - summary, content = parseContent(s, 1, 0) - self.assertEqual(summary, 'blah blah blah blah... blah blah? blah blah blah blah blah. blah blah blah') - self.assertEqual(content, '''\ -blah blah blah blah... blah blah? blah blah blah blah blah. blah blah blah -blah blah blah blah blah blah blah blah blah blah blah! - -issue_tracker@foo.com wrote: -> blah blah blah blahblah blahblah blahblah blah blah blah blah blah blah -> blah blah blah blah blah blah blah blah blah? blah blah blah blah blah -> blah blah blah blah blah blah blah... blah blah blah blah. blah blah -> blah blah blah blah? blah blah blah blah blah blah! blah blah! -> -> ------- -> nosy: userfoo, userken -> _________________________________________________ -> Roundup issue tracker -> issue_tracker@foo.com -> http://foo.com/cgi-bin/roundup.cgi/issue_tracker/''') - - - def testKeepBody(self): - s = ''' -blah blah blah blah... blah blah? blah blah blah blah blah. blah blah blah -blah blah blah blah blah blah blah blah blah blah blah! - -issue_tracker@foo.com wrote: -> blah blah blah blahblah blahblah blahblah blah blah blah blah blah blah -> blah blah blah blah blah blah blah blah blah? blah blah blah blah blah -> blah blah blah blah blah blah blah... blah blah blah blah. blah blah -> blah blah blah blah? blah blah blah blah blah blah! blah blah! -> -> ------- -> nosy: userfoo, userken -> _________________________________________________ -> Roundup issue tracker -> issue_tracker@foo.com -> http://foo.com/cgi-bin/roundup.cgi/issue_tracker/ - --- -blah blah blah signature -userfoo@foo.com -''' - summary, content = parseContent(s, 0, 1) - self.assertEqual(summary, 'blah blah blah blah... blah blah? blah blah blah blah blah. blah blah blah') - self.assertEqual(content, ''' -blah blah blah blah... blah blah? blah blah blah blah blah. blah blah blah -blah blah blah blah blah blah blah blah blah blah blah! - -issue_tracker@foo.com wrote: -> blah blah blah blahblah blahblah blahblah blah blah blah blah blah blah -> blah blah blah blah blah blah blah blah blah? blah blah blah blah blah -> blah blah blah blah blah blah blah... blah blah blah blah. blah blah -> blah blah blah blah? blah blah blah blah blah blah! blah blah! -> -> ------- -> nosy: userfoo, userken -> _________________________________________________ -> Roundup issue tracker -> issue_tracker@foo.com -> http://foo.com/cgi-bin/roundup.cgi/issue_tracker/ - --- -blah blah blah signature -userfoo@foo.com -''') - - - def testSimple(self): - s = '''testing''' - summary, content = parseContent(s, 0, 0) - self.assertEqual(summary, 'testing') - self.assertEqual(content, 'testing') - - def testParagraphs(self): - s = '''testing\n\ntesting\n\ntesting''' - summary, content = parseContent(s, 0, 0) - self.assertEqual(summary, 'testing') - self.assertEqual(content, 'testing\n\ntesting\n\ntesting') - - def testSimpleFollowup(self): - s = '''>hello\ntesting''' - summary, content = parseContent(s, 0, 0) - self.assertEqual(summary, 'testing') - self.assertEqual(content, 'testing') - - def testSimpleFollowupParas(self): - s = '''>hello\ntesting\n\ntesting\n\ntesting''' - summary, content = parseContent(s, 0, 0) - self.assertEqual(summary, 'testing') - self.assertEqual(content, 'testing\n\ntesting\n\ntesting') - - def testEmpty(self): - s = '' - summary, content = parseContent(s, 0, 0) - self.assertEqual(summary, '') - self.assertEqual(content, '') - - def testIndentationSummary(self): - s = ' Four space indent.\n\n Four space indent.\nNo indent.' - summary, content = parseContent(s, 0, 0) - self.assertEqual(summary, ' Four space indent.') - - def testIndentationContent(self): - s = ' Four space indent.\n\n Four space indent.\nNo indent.' - summary, content = parseContent(s, 0, 0) - self.assertEqual(content, s) - -def suite(): - return unittest.makeSuite(MailsplitTestCase, 'test') - - -# vim: set filetype=python ts=4 sw=4 et si