annotate test/test_indexer.py @ 3682:193f316dbbe9

More transitive-property support. - Implemented transitive properties in sort and group specs. Sort/group specs can now be lists of specs. - All regression tests except for one metakit backend test related to metakit having no representation of NULL pass - Fixed more PEP 8 whitespace peeves (and probably introduced some new ones :-) - Moved Proptree from support.py to hyperdb.py due to circular import - Moved some proptree-specific methods from Class to Proptree - Added a test for sorting by ids -> should be numeric sort (which now really works for all backends) - Added "required" attribute to all property classes in hyperdb (e.g., String, Link,...), see Feature Requests [SF#539081] -> factored common stuff to _Type. Note that I also converted to a new-style class when I was at it. Bad: The repr changes for new-style classes which made some SQL backends break (!) because the repr of Multilink is used in the schema storage. Fixed the repr to be independent of the class type. - Added get_required_props to Class. Todo: should also automagically make the key property required... - Add a sort_repr method to property classes. This defines the sort-order. Individual backends may use diffent routines if the outcome is the same. This one has a special case for id properties to make the sorting numeric. Using these methods isn't mandatory in backends as long as the sort-order is correct. - Multilink sorting takes orderprop into account. It used to sort by ids. You can restore the old behaviour by specifying id as the orderprop of the Multilink if you really need that. - If somebody specified a Link or Multilink as orderprop, we sort by labelprop of that class -- not transitively by orderprop. I've resited the tempation to implement recursive orderprop here: There could even be loops if several classes specify a Link or Multilink as the orderprop... - Fixed a bug in Metakit-Backend: When sorting by Links, the backend would do a natural join to the Link class. It would rename the "id" attribute before joining but *not* all the other attributes of the joined class. So in one test-case we had a name-clash with priority.name and status.name when sorting *and* grouping by these attributes. Depending on the order of joining this would produce a name-clash with broken sort-results (and broken display if the original class has an attribute that clashes). I'm now doing the sorting of Links in the generic filter method for the metakit backend. I've left the dead code in the metakit-backend since correctly implementing this in the backend will probably be more efficient. - updated doc/design.html with the new docstring of filter.
author Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
date Mon, 21 Aug 2006 12:19:48 +0000
parents 7728ee93efd2
children 0bf9f8ae7d1b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1 # Copyright (c) 2002 ekit.com Inc (http://www.ekit-inc.com/)
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2 #
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3 # Permission is hereby granted, free of charge, to any person obtaining a copy
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
4 # of this software and associated documentation files (the "Software"), to deal
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
5 # in the Software without restriction, including without limitation the rights
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
6 # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
7 # copies of the Software, and to permit persons to whom the Software is
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
8 # furnished to do so, subject to the following conditions:
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
9 #
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
10 # The above copyright notice and this permission notice shall be included in
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
11 # all copies or substantial portions of the Software.
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
12 #
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
13 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
14 # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
15 # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
16 # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
17 # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
18 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
19 # SOFTWARE.
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
20
3547
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
21 # $Id: test_indexer.py,v 1.10 2006-02-07 04:59:05 richard Exp $
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
22
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
23 import os, unittest, shutil
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
24
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
25 class db:
3546
a4edd24c32be test fixes and checking of indexer overwrites (xapian currently fails)
Richard Jones <richard@users.sourceforge.net>
parents: 3297
diff changeset
26 class config(dict):
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
27 DATABASE = 'test-index'
3546
a4edd24c32be test fixes and checking of indexer overwrites (xapian currently fails)
Richard Jones <richard@users.sourceforge.net>
parents: 3297
diff changeset
28 config = config()
a4edd24c32be test fixes and checking of indexer overwrites (xapian currently fails)
Richard Jones <richard@users.sourceforge.net>
parents: 3297
diff changeset
29 config[('main', 'indexer_stopwords')] = []
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
30
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
31 class IndexerTest(unittest.TestCase):
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
32 def setUp(self):
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
33 if os.path.exists('test-index'):
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
34 shutil.rmtree('test-index')
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
35 os.mkdir('test-index')
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
36 os.mkdir('test-index/files')
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
37 from roundup.backends.indexer_dbm import Indexer
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
38 self.dex = Indexer(db)
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
39 self.dex.load_index()
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
40
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
41 def test_basics(self):
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
42 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
43 self.dex.add_text(('test', '2', 'foo'), 'blah blah the world')
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
44 self.assertEqual(self.dex.find(['world']), [('test', '1', 'foo'),
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
45 ('test', '2', 'foo')])
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
46 self.assertEqual(self.dex.find(['blah']), [('test', '2', 'foo')])
3078
2fee36128471 Fix test to account for changed interface to find().
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2977
diff changeset
47 self.assertEqual(self.dex.find(['blah', 'hello']), [])
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
48
3547
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
49 def test_change(self):
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
50 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
51 self.dex.add_text(('test', '2', 'foo'), 'blah blah the world')
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
52 self.assertEqual(self.dex.find(['world']), [('test', '1', 'foo'),
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
53 ('test', '2', 'foo')])
3546
a4edd24c32be test fixes and checking of indexer overwrites (xapian currently fails)
Richard Jones <richard@users.sourceforge.net>
parents: 3297
diff changeset
54 self.dex.add_text(('test', '1', 'foo'), 'a the hello')
a4edd24c32be test fixes and checking of indexer overwrites (xapian currently fails)
Richard Jones <richard@users.sourceforge.net>
parents: 3297
diff changeset
55 self.assertEqual(self.dex.find(['world']), [('test', '2', 'foo')])
a4edd24c32be test fixes and checking of indexer overwrites (xapian currently fails)
Richard Jones <richard@users.sourceforge.net>
parents: 3297
diff changeset
56
3547
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
57 def test_clear(self):
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
58 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
59 self.dex.add_text(('test', '2', 'foo'), 'blah blah the world')
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
60 self.assertEqual(self.dex.find(['world']), [('test', '1', 'foo'),
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
61 ('test', '2', 'foo')])
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
62 self.dex.add_text(('test', '1', 'foo'), '')
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
63 self.assertEqual(self.dex.find(['world']), [('test', '2', 'foo')])
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
64
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
65 def tearDown(self):
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
66 shutil.rmtree('test-index')
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
67
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
68 class XapianIndexerTest(IndexerTest):
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
69 def setUp(self):
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
70 if os.path.exists('test-index'):
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
71 shutil.rmtree('test-index')
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
72 os.mkdir('test-index')
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
73 from roundup.backends.indexer_xapian import Indexer
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
74 self.dex = Indexer(db)
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
75 def tearDown(self):
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
76 shutil.rmtree('test-index')
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
77
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1090
diff changeset
78 def test_suite():
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1090
diff changeset
79 suite = unittest.TestSuite()
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1090
diff changeset
80 suite.addTest(unittest.makeSuite(IndexerTest))
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
81 try:
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
82 import xapian
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
83 suite.addTest(unittest.makeSuite(XapianIndexerTest))
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
84 except ImportError:
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
85 print "Skipping Xapian indexer tests"
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
86 pass
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1090
diff changeset
87 return suite
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
88
1873
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1090
diff changeset
89 if __name__ == '__main__':
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1090
diff changeset
90 runner = unittest.TextTestRunner()
f63aa57386b0 Backend improvements.
Richard Jones <richard@users.sourceforge.net>
parents: 1090
diff changeset
91 unittest.main(testRunner=runner)
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
92
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
93 # vim: set filetype=python ts=4 sw=4 et si

Roundup Issue Tracker: http://roundup-tracker.org/