Mercurial Repository: p/roundup/code: roundup/hyperdb.py comparison

comparison roundup/hyperdb.py @ 3634:57c66056ffe4

Implemented what I'll call for now "transitive searching"... ...using the filter method. The first idea was mentioned on the roundup-users mailing list: http://article.gmane.org/gmane.comp.bug-tracking.roundup.user/6909 We can now search for items which link transitively to other classes using filter. An example is searching for all items where a certain user has added a message in the last week: db.issue.filter (None, {'messages.author' : '42', 'messages.date' : '.-1w;'}) or more readable (but not exactly semantically equivalent, if you're searching for multiple users in this way it will fail, because string searches are ANDed): {'messages.author.username':'ralf', ... We can even extend this further, look for all items that were changed by users belonging to a certain department (having the same supervisor -- a property that is not in the user class in standard roundup) in the last week, the filterspec would be: {'messages.author.supervisor' : '42', 'messages.date' : '.-1w;'} If anybody wants to suggest another name instead of transitive searching, you're welcome. I've implemented a generic method for this in hyperdb.py -- the backend now implements _filter in this case. With the generic method, anydbm and metakit should work (anydbm is tested, metakit breaks for other reasons). A backend may chose to implement the real transitive filter itself. This was done for rdbms_common.py. It now has an implementation of filter that supports transitive searching by creating one big join in the generated SQL query. I've added several new regression tests to test for the new features. All the tests (not just the new ones) run through on python2.3 and python2.4 with postgres, mysql, sqlite, anydbm -- but metakit was already broken when I started. I've generated a tag before commit called 'rsc_before_transitive_search' and will create the 'after' tag after this commit, so you can merge out my changes if you don't like them -- if you like them I can remove the tags. .-- Ralf

author	Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
date	Sat, 08 Jul 2006 18:28:18 +0000
parents	77ed6c517793
children	53987aa153d2

comparison

equal deleted inserted replaced

-:a292054b4393
+:57c66056ffe4
 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
 # FOR A PARTICULAR PURPOSE.  THE CODE PROVIDED HEREUNDER IS ON AN "AS IS"
 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE,
 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
 #
-# $Id: hyperdb.py,v 1.120 2006-05-06 17:18:03 a1s Exp $
+# $Id: hyperdb.py,v 1.121 2006-07-08 18:28:18 schlatterbeck Exp $
 """Hyperdatabase implementation, especially field types.
 """
 __docformat__ = 'restructuredtext'
 # standard python modules
 import sys, os, time, re, shutil, weakref
 # roundup modules
 import date, password
-from support import ensureParentsExist, PrioList
+from support import ensureParentsExist, PrioList, Proptree
 #
 # Types
 #
 class String:
 db.issue.find(messages={'1':1,'3':1}, files={'7':1})
 """
 raise NotImplementedError
+def _filter(self, search_matches, filterspec, sort=(None,None),
+group=(None,None)):
+"""For some backends this implements the non-transitive
+search, for more information see the filter method.
+"""
+raise NotImplementedError
+def _proptree (self, filterspec) :
+"""Build a tree of all transitive properties in the given
+filterspec.
+"""
+proptree = Proptree (self.db, self, '', self.getprops ())
+for key, v in filterspec.iteritems () :
+keys = key.split ('.')
+p = proptree
+for k in keys :
+p = p.append (k)
+p.val = v
+return proptree
+def _propsearch (self, search_matches, proptree, sort, group) :
+""" Recursively search for the given properties in proptree.
+Once all properties are non-transitive, the search generates a
+simple _filter call which does the real work
+"""
+for p in proptree.children :
+if not p.children : continue
+p.val = p.cls._propsearch (None, p, (None, None), (None, None))
+filterspec = dict ([(p.name, p.val) for p in proptree.children])
+return self._filter (search_matches, filterspec, sort, group)
 def filter(self, search_matches, filterspec, sort=(None,None),
 group=(None,None)):
 """Return a list of the ids of the active nodes in this class that
 match the 'filter' spec, sorted by the group spec and then the
 sort spec.
 "filterspec" is {propname: value(s)}
+Note that now the propname in filterspec may be transitive,
+i.e., it may contain properties of the form link.link.link.name,
+e.g. you can search for all issues where a message was added by
+a certain user in the last week with a filterspec of
+{'messages.author' : '42', 'messages.creation' : '.-1w;'}
 "sort" and "group" are (dir, prop) where dir is '+', '-' or None
 and prop is a prop name or None
 "search_matches" is {nodeid: marker}
 The filter must match all properties specificed. If the property
 value to match is a list:
 1. String properties must match all elements in the list, and
 2. Other properties must match any of the elements in the list.
-"""
-raise NotImplementedError
+Implementation note:
+This implements a non-optimized version of Transitive search
+using _filter implemented in a backend class. A more efficient
+version can be implemented in the individual backends -- e.g.,
+an SQL backen will want to create a single SQL statement and
+override the filter method instead of implementing _filter.
+"""
+proptree = self._proptree (filterspec)
+return self._propsearch (search_matches, proptree, sort, group)
 def count(self):
 """Get the number of nodes in this class.
 If the returned integer is 'numnodes', the ids of all the nodes

Mercurial > p > roundup > code

comparison roundup/hyperdb.py @ 3634:57c66056ffe4