view test/test_indexer.py @ 3634:57c66056ffe4

Implemented what I'll call for now "transitive searching"... ...using the filter method. The first idea was mentioned on the roundup-users mailing list: http://article.gmane.org/gmane.comp.bug-tracking.roundup.user/6909 We can now search for items which link transitively to other classes using filter. An example is searching for all items where a certain user has added a message in the last week: db.issue.filter (None, {'messages.author' : '42', 'messages.date' : '.-1w;'}) or more readable (but not exactly semantically equivalent, if you're searching for multiple users in this way it will fail, because string searches are ANDed): {'messages.author.username':'ralf', ... We can even extend this further, look for all items that were changed by users belonging to a certain department (having the same supervisor -- a property that is not in the user class in standard roundup) in the last week, the filterspec would be: {'messages.author.supervisor' : '42', 'messages.date' : '.-1w;'} If anybody wants to suggest another name instead of transitive searching, you're welcome. I've implemented a generic method for this in hyperdb.py -- the backend now implements _filter in this case. With the generic method, anydbm and metakit should work (anydbm is tested, metakit breaks for other reasons). A backend may chose to implement the real transitive filter itself. This was done for rdbms_common.py. It now has an implementation of filter that supports transitive searching by creating one big join in the generated SQL query. I've added several new regression tests to test for the new features. All the tests (not just the new ones) run through on python2.3 and python2.4 with postgres, mysql, sqlite, anydbm -- but metakit was already broken when I started. I've generated a tag before commit called 'rsc_before_transitive_search' and will create the 'after' tag after this commit, so you can merge out my changes if you don't like them -- if you like them I can remove the tags. .-- Ralf
author Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
date Sat, 08 Jul 2006 18:28:18 +0000
parents 7728ee93efd2
children 0bf9f8ae7d1b
line wrap: on
line source

# Copyright (c) 2002 ekit.com Inc (http://www.ekit-inc.com/)
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
#   The above copyright notice and this permission notice shall be included in
#   all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

# $Id: test_indexer.py,v 1.10 2006-02-07 04:59:05 richard Exp $

import os, unittest, shutil

class db:
    class config(dict):
        DATABASE = 'test-index'
    config = config()
    config[('main', 'indexer_stopwords')] = []

class IndexerTest(unittest.TestCase):
    def setUp(self):
        if os.path.exists('test-index'):
            shutil.rmtree('test-index')
        os.mkdir('test-index')
        os.mkdir('test-index/files')
        from roundup.backends.indexer_dbm import Indexer
        self.dex = Indexer(db)
        self.dex.load_index()

    def test_basics(self):
        self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
        self.dex.add_text(('test', '2', 'foo'), 'blah blah the world')
        self.assertEqual(self.dex.find(['world']), [('test', '1', 'foo'),
                                                    ('test', '2', 'foo')])
        self.assertEqual(self.dex.find(['blah']), [('test', '2', 'foo')])
        self.assertEqual(self.dex.find(['blah', 'hello']), [])

    def test_change(self):
        self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
        self.dex.add_text(('test', '2', 'foo'), 'blah blah the world')
        self.assertEqual(self.dex.find(['world']), [('test', '1', 'foo'),
                                                    ('test', '2', 'foo')])
        self.dex.add_text(('test', '1', 'foo'), 'a the hello')
        self.assertEqual(self.dex.find(['world']), [('test', '2', 'foo')])

    def test_clear(self):
        self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
        self.dex.add_text(('test', '2', 'foo'), 'blah blah the world')
        self.assertEqual(self.dex.find(['world']), [('test', '1', 'foo'),
                                                    ('test', '2', 'foo')])
        self.dex.add_text(('test', '1', 'foo'), '')
        self.assertEqual(self.dex.find(['world']), [('test', '2', 'foo')])

    def tearDown(self):
        shutil.rmtree('test-index')

class XapianIndexerTest(IndexerTest):
    def setUp(self):
        if os.path.exists('test-index'):
            shutil.rmtree('test-index')
        os.mkdir('test-index')
        from roundup.backends.indexer_xapian import Indexer
        self.dex = Indexer(db)
    def tearDown(self):
        shutil.rmtree('test-index')

def test_suite():
    suite = unittest.TestSuite()
    suite.addTest(unittest.makeSuite(IndexerTest))
    try:
        import xapian
        suite.addTest(unittest.makeSuite(XapianIndexerTest))
    except ImportError:
        print "Skipping Xapian indexer tests"
        pass
    return suite

if __name__ == '__main__':
    runner = unittest.TextTestRunner()
    unittest.main(testRunner=runner)

# vim: set filetype=python ts=4 sw=4 et si

Roundup Issue Tracker: http://roundup-tracker.org/