Mercurial > p > roundup > code

annotate test/test_indexer.py @ 7478:e8d2a4bca16a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Update index. Add "track your issues your way" tag line and... Add description of benefits to some of the noteworthy changes. Add GTD "thing management" from old email in the 1.6 time period. Reverse order of fast gratification so source directory is first then venv install. Match the first line that says you don't have to install.
author John Rouillard <rouilj@ieee.org>
date Sun, 11 Jun 2023 21:32:46 -0400
parents 67e7225c4343
children d17e57220a62
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1 # Copyright (c) 2002 ekit.com Inc (http://www.ekit-inc.com/)
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2 #
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3 # Permission is hereby granted, free of charge, to any person obtaining a copy
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
4 # of this software and associated documentation files (the "Software"), to deal
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
5 # in the Software without restriction, including without limitation the rights
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
6 # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
7 # copies of the Software, and to permit persons to whom the Software is
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
8 # furnished to do so, subject to the following conditions:
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
9 #
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
10 # The above copyright notice and this permission notice shall be included in
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
11 # all copies or substantial portions of the Software.
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
12 #
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
13 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
14 # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
15 # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
16 # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
17 # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
18 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
19 # SOFTWARE.
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
20
7070
c6b2534a58a9 Compatibility: unittest.mock vs. only mock
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6915
diff changeset
21 import os, sys, unittest, shutil
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
22
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
23 import pytest
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
24 from roundup.backends import get_backend, have_backend
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
25 from roundup.backends.indexer_rdbms import Indexer
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
26 from roundup.backends.indexer_common import get_indexer
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
27
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
28 from roundup.cgi.exceptions import IndexerQueryError
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
29
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
30 # borrow from other tests
5388
d26921b851c3 Python 3 preparation: make relative imports explicit.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5109
diff changeset
31 from .db_test_base import setupSchema, config
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
32 from .test_postgresql import postgresqlOpener, skip_postgresql
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
33 from .test_mysql import mysqlOpener, skip_mysql
5388
d26921b851c3 Python 3 preparation: make relative imports explicit.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5109
diff changeset
34 from .test_sqlite import sqliteOpener
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
35 from .test_anydbm import anydbmOpener
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
36
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
37 try:
7070
c6b2534a58a9 Compatibility: unittest.mock vs. only mock
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6915
diff changeset
38 from unittest import mock
c6b2534a58a9 Compatibility: unittest.mock vs. only mock
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6915
diff changeset
39 except ImportError:
c6b2534a58a9 Compatibility: unittest.mock vs. only mock
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6915
diff changeset
40 import mock
c6b2534a58a9 Compatibility: unittest.mock vs. only mock
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6915
diff changeset
41
c6b2534a58a9 Compatibility: unittest.mock vs. only mock
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6915
diff changeset
42 try:
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
43 import xapian
5038
c977f3530944 Work-around for pytest.mark.skipif() bug
John Kristensen <john@jerrykan.com>
parents: 5037
diff changeset
44 skip_xapian = lambda func, *args, **kwargs: func
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
45 except ImportError:
5109
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
46 # FIX: workaround for a bug in pytest.mark.skip():
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
47 # https://github.com/pytest-dev/pytest/issues/568
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
48 from .pytest_patcher import mark_class
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
49 skip_xapian = mark_class(pytest.mark.skip(
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
50 "Skipping Xapian indexer tests: 'xapian' not installed"))
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
51
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
52 try:
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
53 import whoosh
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
54 skip_whoosh = lambda func, *args, **kwargs: func
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
55 except ImportError:
5109
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
56 # FIX: workaround for a bug in pytest.mark.skip():
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
57 # https://github.com/pytest-dev/pytest/issues/568
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
58 from .pytest_patcher import mark_class
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
59 skip_whoosh = mark_class(pytest.mark.skip(
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
60 "Skipping Whoosh indexer tests: 'whoosh' not installed"))
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
61
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
62 class db:
3546
a4edd24c32be test fixes and checking of indexer overwrites (xapian currently fails)
Richard Jones <richard@users.sourceforge.net>
parents: 3297
diff changeset
63 class config(dict):
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
64 DATABASE = 'test-index'
3546
a4edd24c32be test fixes and checking of indexer overwrites (xapian currently fails)
Richard Jones <richard@users.sourceforge.net>
parents: 3297
diff changeset
65 config = config()
a4edd24c32be test fixes and checking of indexer overwrites (xapian currently fails)
Richard Jones <richard@users.sourceforge.net>
parents: 3297
diff changeset
66 config[('main', 'indexer_stopwords')] = []
6353
9d209d2b34ae Add indexer_language to change stemmer for xapian FTS indexer
John Rouillard <rouilj@ieee.org>
parents: 5964
diff changeset
67 config[('main', 'indexer_language')] = "english"
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
68
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
69 class IndexerTest(anydbmOpener, unittest.TestCase):
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
70
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
71 indexer_name = "native"
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
72
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
73 def setUp(self):
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
74 # remove previous test, ignore errors
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
75 if os.path.exists(config.DATABASE):
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
76 shutil.rmtree(config.DATABASE)
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
77 self.db = self.module.Database(config, 'admin')
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
78
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
79 if os.path.exists('test-index'):
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
80 shutil.rmtree('test-index')
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
81 os.mkdir('test-index')
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
82 os.mkdir('test-index/files')
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
83 from roundup.backends.indexer_dbm import Indexer
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
84 self.dex = Indexer(db)
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
85 self.dex.load_index()
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
86
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
87 def assertSeqEqual(self, s1, s2):
4102
dcca66d56815 fix unit test compatibility
Richard Jones <richard@users.sourceforge.net>
parents: 4016
diff changeset
88 # First argument is the db result we're testing, second is the
dcca66d56815 fix unit test compatibility
Richard Jones <richard@users.sourceforge.net>
parents: 4016
diff changeset
89 # desired result. Some db results don't have iterable rows, so we
dcca66d56815 fix unit test compatibility
Richard Jones <richard@users.sourceforge.net>
parents: 4016
diff changeset
90 # have to work around that.
4015
6eec11b197aa fix for indexer-test:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4008
diff changeset
91 # Also work around some dbs not returning items in the expected
4102
dcca66d56815 fix unit test compatibility
Richard Jones <richard@users.sourceforge.net>
parents: 4016
diff changeset
92 # order.
dcca66d56815 fix unit test compatibility
Richard Jones <richard@users.sourceforge.net>
parents: 4016
diff changeset
93 s1 = list([tuple([r[n] for n in range(len(r))]) for r in s1])
4015
6eec11b197aa fix for indexer-test:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4008
diff changeset
94 s1.sort()
4102
dcca66d56815 fix unit test compatibility
Richard Jones <richard@users.sourceforge.net>
parents: 4016
diff changeset
95 if s1 != s2:
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
96 self.fail('contents of %r != %r'%(s1, s2))
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
97
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
98 def test_basics(self):
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
99 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
100 self.dex.add_text(('test', '2', 'foo'), 'blah blah the world')
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
101 self.assertSeqEqual(self.dex.find(['world']), [('test', '1', 'foo'),
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
102 ('test', '2', 'foo')])
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
103 self.assertSeqEqual(self.dex.find(['blah']), [('test', '2', 'foo')])
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
104 self.assertSeqEqual(self.dex.find(['blah', 'hello']), [])
6915
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
105 self.assertSeqEqual(self.dex.find([]), [])
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
106
3547
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
107 def test_change(self):
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
108 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
109 self.dex.add_text(('test', '2', 'foo'), 'blah blah the world')
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
110 self.assertSeqEqual(self.dex.find(['world']), [('test', '1', 'foo'),
3547
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
111 ('test', '2', 'foo')])
3546
a4edd24c32be test fixes and checking of indexer overwrites (xapian currently fails)
Richard Jones <richard@users.sourceforge.net>
parents: 3297
diff changeset
112 self.dex.add_text(('test', '1', 'foo'), 'a the hello')
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
113 self.assertSeqEqual(self.dex.find(['world']), [('test', '2', 'foo')])
3546
a4edd24c32be test fixes and checking of indexer overwrites (xapian currently fails)
Richard Jones <richard@users.sourceforge.net>
parents: 3297
diff changeset
114
3547
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
115 def test_clear(self):
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
116 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
117 self.dex.add_text(('test', '2', 'foo'), 'blah blah the world')
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
118 self.assertSeqEqual(self.dex.find(['world']), [('test', '1', 'foo'),
3547
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
119 ('test', '2', 'foo')])
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
120 self.dex.add_text(('test', '1', 'foo'), '')
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
121 self.assertSeqEqual(self.dex.find(['world']), [('test', '2', 'foo')])
3547
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
122
6912
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
123 def test_get_indexer_specified(self):
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
124 """Specify an indexer back end and make sure it's returned"""
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
125 def class_name_of(object):
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
126 """ take and object and return just the class name.
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
127 So in:
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
128
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
129 return the class name before "at".
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
130
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
131 """
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
132 return(str(object).split()[0])
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
133
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
134 old_indexer = self.db.config['INDEXER']
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
135 self.db.config['INDEXER'] = self.indexer_name
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
136
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
137 self.assertEqual(class_name_of(self.dex),
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
138 class_name_of(get_indexer(self.db.config, self.db)))
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
139
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
140 self.db.config['INDEXER'] = old_indexer
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
141
4251
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
142 def test_stopwords(self):
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
143 """Test that we can find a text with a stopword in it."""
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
144 stopword = "with"
5649
f8893e1cde0d assert_ is depricated. Replacing with assertTrue to reduce logs in travisci.
John Rouillard <rouilj@ieee.org>
parents: 5388
diff changeset
145 self.assertTrue(self.dex.is_stopword(stopword.upper()))
4251
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
146 self.dex.add_text(('test', '1', 'bar'), '%s hello world' % stopword)
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
147 self.dex.add_text(('test', '2', 'bar'), 'blah a %s world' % stopword)
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
148 self.dex.add_text(('test', '3', 'bar'), 'blah Blub river')
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
149 self.dex.add_text(('test', '4', 'bar'), 'blah river %s' % stopword)
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
150 self.assertSeqEqual(self.dex.find(['with','world']),
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
151 [('test', '1', 'bar'),
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
152 ('test', '2', 'bar')])
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
153 def test_extremewords(self):
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
154 """Testing too short or too long words."""
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
155
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
156 # skip this for FTS test
6604
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
157 if ( isinstance(self,sqliteFtsIndexerTest) or
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
158 isinstance(self,postgresqlFtsIndexerTest)):
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
159 pytest.skip("extremewords not tested for native FTS backends")
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
160
4251
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
161 short = "b"
6599
39189dd94f2c issue2551189 - increase size of words in full text index.
John Rouillard <rouilj@ieee.org>
parents: 6588
diff changeset
162 long = "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"
4251
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
163 self.dex.add_text(('test', '1', 'a'), '%s hello world' % short)
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
164 self.dex.add_text(('test', '2', 'a'), 'blah a %s world' % short)
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
165 self.dex.add_text(('test', '3', 'a'), 'blah Blub river')
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
166 self.dex.add_text(('test', '4', 'a'), 'blah river %s %s'
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
167 % (short, long))
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
168 self.assertSeqEqual(self.dex.find([short,'world', long, short]),
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
169 [('test', '1', 'a'),
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
170 ('test', '2', 'a')])
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
171 self.assertSeqEqual(self.dex.find([long]),[])
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
172
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
173 # special test because some faulty code indexed length(word)>=2
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
174 # but only considered length(word)>=3 to be significant
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
175 self.dex.add_text(('test', '5', 'a'), 'blah py %s %s'
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
176 % (short, long))
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
177 self.assertSeqEqual(self.dex.find(["py"]), [('test', '5', 'a')])
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
178
6604
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
179 def test_casesensitivity(self):
4251
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
180 """Test if searches are case-in-sensitive."""
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
181 self.dex.add_text(('test', '1', 'a'), 'aaaa bbbb')
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
182 self.dex.add_text(('test', '2', 'a'), 'aAaa BBBB')
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
183 self.assertSeqEqual(self.dex.find(['aaaa']),
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
184 [('test', '1', 'a'),
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
185 ('test', '2', 'a')])
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
186 self.assertSeqEqual(self.dex.find(['BBBB']),
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
187 [('test', '1', 'a'),
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
188 ('test', '2', 'a')])
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
189
4314
b41a033bffcc - add a small word-splitting test for the indexers...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4251
diff changeset
190 def test_wordsplitting(self):
b41a033bffcc - add a small word-splitting test for the indexers...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4251
diff changeset
191 """Test if word splitting works."""
b41a033bffcc - add a small word-splitting test for the indexers...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4251
diff changeset
192 self.dex.add_text(('test', '1', 'a'), 'aaaa-aaa bbbb*bbb')
b41a033bffcc - add a small word-splitting test for the indexers...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4251
diff changeset
193 self.dex.add_text(('test', '2', 'a'), 'aaaA-aaa BBBB*BBB')
b41a033bffcc - add a small word-splitting test for the indexers...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4251
diff changeset
194 for k in 'aaaa', 'aaa', 'bbbb', 'bbb':
b41a033bffcc - add a small word-splitting test for the indexers...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4251
diff changeset
195 self.assertSeqEqual(self.dex.find([k]),
b41a033bffcc - add a small word-splitting test for the indexers...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4251
diff changeset
196 [('test', '1', 'a'), ('test', '2', 'a')])
b41a033bffcc - add a small word-splitting test for the indexers...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4251
diff changeset
197
4841
3ff1a288fb9c issue2550583, issue2550635 Do not limit results with Xapian indexer
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4570
diff changeset
198 def test_manyresults(self):
3ff1a288fb9c issue2550583, issue2550635 Do not limit results with Xapian indexer
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4570
diff changeset
199 """Test if searches find many results."""
3ff1a288fb9c issue2550583, issue2550635 Do not limit results with Xapian indexer
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4570
diff changeset
200 for i in range(123):
3ff1a288fb9c issue2550583, issue2550635 Do not limit results with Xapian indexer
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4570
diff changeset
201 self.dex.add_text(('test', str(i), 'many'), 'many')
3ff1a288fb9c issue2550583, issue2550635 Do not limit results with Xapian indexer
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4570
diff changeset
202 self.assertEqual(len(self.dex.find(['many'])), 123)
3ff1a288fb9c issue2550583, issue2550635 Do not limit results with Xapian indexer
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4570
diff changeset
203
5960
0db2621b6fee Add test for issue1344046 and maybe issue1195739
John Rouillard <rouilj@ieee.org>
parents: 5649
diff changeset
204 def test_unicode(self):
0db2621b6fee Add test for issue1344046 and maybe issue1195739
John Rouillard <rouilj@ieee.org>
parents: 5649
diff changeset
205 """Test with unicode words. see:
0db2621b6fee Add test for issue1344046 and maybe issue1195739
John Rouillard <rouilj@ieee.org>
parents: 5649
diff changeset
206 https://issues.roundup-tracker.org/issue1344046"""
5962
6137ea845438 Fix russian search/string
John Rouillard <rouilj@ieee.org>
parents: 5961
diff changeset
207 russian=u'\u0440\u0443\u0441\u0441\u043a\u0438\u0439 \u0442\u0435\u043a\u0441\u0442Spr\xfcnge'
5961
f6c58a7b535c Add a russian string for unicode testing of indexer
John Rouillard <rouilj@ieee.org>
parents: 5960
diff changeset
208 german=u'Spr\xfcnge'
f6c58a7b535c Add a russian string for unicode testing of indexer
John Rouillard <rouilj@ieee.org>
parents: 5960
diff changeset
209 self.dex.add_text(('test', '1', 'a'), german )
f6c58a7b535c Add a russian string for unicode testing of indexer
John Rouillard <rouilj@ieee.org>
parents: 5960
diff changeset
210 self.dex.add_text(('test', '2', 'a'), russian + u' ' + german )
f6c58a7b535c Add a russian string for unicode testing of indexer
John Rouillard <rouilj@ieee.org>
parents: 5960
diff changeset
211
f6c58a7b535c Add a russian string for unicode testing of indexer
John Rouillard <rouilj@ieee.org>
parents: 5960
diff changeset
212 self.assertSeqEqual(self.dex.find([ u'Spr\xfcnge']),
f6c58a7b535c Add a russian string for unicode testing of indexer
John Rouillard <rouilj@ieee.org>
parents: 5960
diff changeset
213 [('test', '1', 'a'), ('test', '2', 'a')])
5962
6137ea845438 Fix russian search/string
John Rouillard <rouilj@ieee.org>
parents: 5961
diff changeset
214 self.assertSeqEqual(self.dex.find([u'\u0440\u0443\u0441\u0441\u043a\u0438\u0439']),
5961
f6c58a7b535c Add a russian string for unicode testing of indexer
John Rouillard <rouilj@ieee.org>
parents: 5960
diff changeset
215 [('test', '2', 'a')])
6915
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
216
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
217 def testNullChar(self):
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
218 """Test with null char in string. Postgres FTS will not index
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
219 it will just ignore string for now.
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
220 """
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
221 string="\x00\x01fred\x255"
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
222 self.dex.add_text(('test', '1', 'a'), string)
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
223 self.assertSeqEqual(self.dex.find(string), [])
5960
0db2621b6fee Add test for issue1344046 and maybe issue1195739
John Rouillard <rouilj@ieee.org>
parents: 5649
diff changeset
224
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
225 def tearDown(self):
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
226 shutil.rmtree('test-index')
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
227 if hasattr(self, 'db'):
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
228 self.db.close()
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
229 if os.path.exists(config.DATABASE):
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
230 shutil.rmtree(config.DATABASE)
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
231
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
232 @skip_whoosh
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
233 class WhooshIndexerTest(IndexerTest):
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
234
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
235 indexer_name = "whoosh"
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
236
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
237 def setUp(self):
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
238 IndexerTest.setUp(self)
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
239
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
240 if os.path.exists('test-index'):
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
241 shutil.rmtree('test-index')
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
242 os.mkdir('test-index')
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
243 from roundup.backends.indexer_whoosh import Indexer
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
244 self.dex = Indexer(db)
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
245 def tearDown(self):
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
246 IndexerTest.tearDown(self)
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
247
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
248 @skip_xapian
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
249 class XapianIndexerTest(IndexerTest):
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
250
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
251 indexer_name = "xapian"
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
252
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
253 def setUp(self):
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
254 IndexerTest.setUp(self)
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
255
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
256 if os.path.exists('test-index'):
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
257 shutil.rmtree('test-index')
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
258 os.mkdir('test-index')
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
259 from roundup.backends.indexer_xapian import Indexer
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
260 self.dex = Indexer(db)
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
261 def tearDown(self):
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
262 IndexerTest.tearDown(self)
5033
63c79c0992ae Update tests to work with py.test
John Kristensen <john@jerrykan.com>
parents: 4841
diff changeset
263
6915
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
264 class Get_IndexerTest(anydbmOpener, unittest.TestCase):
6912
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
265
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
266 def setUp(self):
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
267 # remove previous test, ignore errors
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
268 if os.path.exists(config.DATABASE):
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
269 shutil.rmtree(config.DATABASE)
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
270 self.db = self.module.Database(config, 'admin')
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
271 # this is the default, but set it in case default
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
272 # changes in future
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
273 self.db.config['INDEXER'] = ''
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
274
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
275 def tearDown(self):
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
276 if hasattr(self, 'db'):
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
277 self.db.close()
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
278 if os.path.exists(config.DATABASE):
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
279 shutil.rmtree(config.DATABASE)
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
280
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
281 @skip_xapian
6915
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
282 def test_xapian_autoselect(self):
6912
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
283 indexer = get_indexer(self.db.config, self.db)
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
284 self.assertIn('roundup.backends.indexer_xapian.Indexer', str(indexer))
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
285
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
286 @skip_whoosh
6915
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
287 def test_whoosh_autoselect(self):
6912
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
288 with mock.patch.dict('sys.modules',
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
289 {'roundup.backends.indexer_xapian': None}):
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
290 indexer = get_indexer(self.db.config, self.db)
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
291 self.assertIn('roundup.backends.indexer_whoosh.Indexer', str(indexer))
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
292
6915
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
293 def test_native_autoselect(self):
6912
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
294 with mock.patch.dict('sys.modules',
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
295 {'roundup.backends.indexer_xapian': None,
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
296 'roundup.backends.indexer_whoosh': None}):
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
297 indexer = get_indexer(self.db.config, self.db)
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
298 self.assertIn('roundup.backends.indexer_dbm.Indexer', str(indexer))
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
299
6915
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
300 def test_invalid_indexer(self):
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
301 """There is code at the end of indexer_common::get_indexer() to
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
302 raise an AssertionError if the indexer name is invalid.
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
303 This should never be triggered. If it is, it means that
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
304 the code in configure.py that validates indexer names
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
305 allows a name through that get_indexer can't handle.
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
306
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
307 Simulate that failure and make sure that the
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
308 AssertionError is raised.
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
309
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
310 """
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
311
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
312 with self.assertRaises(ValueError) as cm:
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
313 self.db.config['INDEXER'] = 'no_such_indexer'
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
314
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
315 # mangle things so we can test AssertionError at end
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
316 # get_indexer()
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
317 from roundup.configuration import IndexerOption
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
318 IndexerOption.allowed.append("unrecognized_indexer")
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
319 self.db.config['INDEXER'] = "unrecognized_indexer"
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
320
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
321 with self.assertRaises(AssertionError) as cm:
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
322 indexer = get_indexer(self.db.config, self.db)
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
323
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
324 # unmangle state
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
325 IndexerOption.allowed.pop()
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
326 self.assertNotIn("unrecognized_indexer", IndexerOption.allowed)
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
327 self.db.config['INDEXER'] = ""
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
328
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
329 def test_unsupported_by_db(self):
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
330 """This requires that the db associated with the test
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
331 is not sqlite or postgres. anydbm works fine to trigger
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
332 the error.
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
333 """
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
334 self.db.config['INDEXER'] = 'native-fts'
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
335 with self.assertRaises(AssertionError) as cm:
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
336 get_indexer(self.db.config, self.db)
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
337
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
338 self.assertIn("native-fts", cm.exception.args[0])
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
339 self.db.config['INDEXER'] = ''
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
340
5033
63c79c0992ae Update tests to work with py.test
John Kristensen <john@jerrykan.com>
parents: 4841
diff changeset
341 class RDBMSIndexerTest(object):
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
342 def setUp(self):
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
343 # remove previous test, ignore errors
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
344 if os.path.exists(config.DATABASE):
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
345 shutil.rmtree(config.DATABASE)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
346 self.db = self.module.Database(config, 'admin')
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
347 self.dex = Indexer(self.db)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
348 def tearDown(self):
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
349 if hasattr(self, 'db'):
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
350 self.db.close()
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
351 if os.path.exists(config.DATABASE):
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
352 shutil.rmtree(config.DATABASE)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
353
5033
63c79c0992ae Update tests to work with py.test
John Kristensen <john@jerrykan.com>
parents: 4841
diff changeset
354
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
355 @skip_postgresql
5033
63c79c0992ae Update tests to work with py.test
John Kristensen <john@jerrykan.com>
parents: 4841
diff changeset
356 class postgresqlIndexerTest(postgresqlOpener, RDBMSIndexerTest, IndexerTest):
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
357 def setUp(self):
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
358 postgresqlOpener.setUp(self)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
359 RDBMSIndexerTest.setUp(self)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
360 def tearDown(self):
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
361 RDBMSIndexerTest.tearDown(self)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
362 postgresqlOpener.tearDown(self)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
363
6604
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
364
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
365 @skip_postgresql
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
366 class postgresqlFtsIndexerTest(postgresqlOpener, RDBMSIndexerTest, IndexerTest):
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
367
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
368 indexer_name = "native-fts"
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
369
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
370 def setUp(self):
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
371 postgresqlOpener.setUp(self)
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
372 RDBMSIndexerTest.setUp(self)
6604
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
373 from roundup.backends.indexer_postgresql_fts import Indexer
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
374 self.dex = Indexer(self.db)
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
375 self.dex.db = self.db
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
376
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
377 def tearDown(self):
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
378 RDBMSIndexerTest.tearDown(self)
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
379 postgresqlOpener.tearDown(self)
6604
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
380
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
381 def test_websearch_syntax(self):
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
382 """Test searches using websearch_to_tsquery. These never throw
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
383 errors regardless of how wacky the input.
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
384 """
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
385
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
386 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
387 self.dex.add_text(('test', '2', 'foo'), 'helh blah blah the world')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
388 self.dex.add_text(('test', '3', 'foo'), 'blah hello the world')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
389 self.dex.add_text(('test', '4', 'foo'), 'hello blah blech the world')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
390 self.dex.add_text(('test', '5', 'foo'), 'a car drove')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
391 self.dex.add_text(('test', '6', 'foo'), 'a car driving itself')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
392 self.dex.add_text(('test', '7', 'foo'), "let's drive in the car")
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
393 self.dex.add_text(('test', '8', 'foo'), 'a drive-in movie')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
394
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
395 # test two separate words for sanity
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
396 self.assertSeqEqual(self.dex.find(['"hello" "world"']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
397 [('test', '1', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
398 ('test', '3', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
399 ('test', '4', 'foo')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
400 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
401 # now check the phrase
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
402 self.assertSeqEqual(self.dex.find(['"hello world"']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
403 [('test', '1', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
404 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
405
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
406 # test negation
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
407 self.assertSeqEqual(self.dex.find(['hello world -blech']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
408 [('test', '1', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
409 ('test', '3', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
410 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
411
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
412 # phrase negation
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
413 self.assertSeqEqual(self.dex.find(['hello world -"blah hello"']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
414 [('test', '1', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
415 ('test', '4', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
416 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
417
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
418 # test without or
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
419 self.assertSeqEqual(self.dex.find(['blah blech']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
420 [('test', '4', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
421 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
422
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
423 # test with or
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
424 self.assertSeqEqual(self.dex.find(['blah or blech']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
425 [ ('test', '2', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
426 ('test', '3', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
427 ('test', '4', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
428 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
429
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
430 # stemmer test for english
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
431 self.assertSeqEqual(self.dex.find(['ts:drive']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
432 [('test', '6', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
433 ('test', '7', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
434 ('test', '8', 'foo')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
435 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
436
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
437 # stemmer is not disabled by quotes 8-(
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
438 self.assertSeqEqual(self.dex.find(['ts:"drive"']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
439 [('test', '6', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
440 ('test', '7', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
441 ('test', '8', 'foo')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
442 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
443
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
444
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
445 # this is missing ts: at the start, so uses the websearch
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
446 # parser. We search for operator characters and wanr the user
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
447 # Otherwise "hello <-> world" is the same as "hello world"
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
448 # and is not a phrase search.
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
449 with self.assertRaises(IndexerQueryError) as ctx:
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
450 self.dex.find(['hello <-> world'])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
451
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
452 self.assertIn('do a tsquery search', ctx.exception.args[0])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
453
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
454 def test_tsquery_syntax(self):
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
455 """Because websearch_to_tsquery doesn't allow prefix searches,
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
456 near searches with any value except 1 (phrase search), allow
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
457 use of to_tsquery by prefixing the search term wih ts:.
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
458
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
459 However, unlike websearch_to_tsquery, this will throw a
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
460 psycopg2.errors.SyntaxError on bad input. SyntaxError is
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
461 re-raised as IndexerQueryError. But it makes a bunch of
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
462 useful expert functionality available.
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
463
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
464 """
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
465
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
466 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
467 self.dex.add_text(('test', '2', 'foo'), 'helh blah blah the world')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
468 self.dex.add_text(('test', '3', 'foo'), 'blah hello the world')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
469 self.dex.add_text(('test', '4', 'foo'), 'hello blah blech the world')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
470 self.dex.add_text(('test', '5', 'foo'), 'a car drove')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
471 self.dex.add_text(('test', '6', 'foo'), 'a car driving itself')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
472 self.dex.add_text(('test', '7', 'foo'), "let's drive in the car")
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
473 self.dex.add_text(('test', '8', 'foo'), 'a drive-in movie')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
474 self.dex.db.commit()
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
475
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
476 # test two separate words for sanity
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
477 self.assertSeqEqual(self.dex.find(['ts:hello & world']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
478 [('test', '1', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
479 ('test', '3', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
480 ('test', '4', 'foo')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
481 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
482 # now check the phrase
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
483 self.assertSeqEqual(self.dex.find(['ts:hello <-> world']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
484 [('test', '1', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
485 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
486
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
487 # test negation
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
488 self.assertSeqEqual(self.dex.find(['ts:hello & world & !blech']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
489 [('test', '1', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
490 ('test', '3', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
491 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
492
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
493 self.assertSeqEqual(self.dex.find(
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
494 ['ts:hello & world & !(blah <-> hello)']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
495 [('test', '1', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
496 ('test', '4', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
497 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
498
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
499 # test without or
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
500 self.assertSeqEqual(self.dex.find(['ts:blah & blech']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
501 [('test', '4', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
502 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
503
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
504 # test with or
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
505 self.assertSeqEqual(self.dex.find(['ts:blah | blech']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
506 [ ('test', '2', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
507 ('test', '3', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
508 ('test', '4', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
509 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
510 # stemmer test for english
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
511 self.assertSeqEqual(self.dex.find(['ts:drive']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
512 [('test', '6', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
513 ('test', '7', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
514 ('test', '8', 'foo')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
515 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
516
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
517 # stemmer is not disabled by quotes 8-(
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
518 self.assertSeqEqual(self.dex.find(['ts:"drive"']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
519 [('test', '6', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
520 ('test', '7', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
521 ('test', '8', 'foo')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
522 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
523
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
524
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
525 # test with syntax error
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
526 with self.assertRaises(IndexerQueryError) as ctx:
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
527 self.dex.find(['ts:blah blech'])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
528
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
529 self.assertEqual(ctx.exception.args[0],
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
530 'syntax error in tsquery: "blah blech"\n')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
531
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
532 # now check the phrase Note unlike sqlite, order matters,
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
533 # hello must come first.
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
534 self.assertSeqEqual(self.dex.find(['ts:hello <-> world']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
535 [('test', '1', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
536 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
537
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
538 # now check the phrase with explicitly 1 intervening item
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
539 self.assertSeqEqual(self.dex.find(['ts:hello <2> world']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
540 [('test', '3', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
541 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
542 # now check the phrase with near explicitly 1 or 3 intervening items
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
543 self.assertSeqEqual(self.dex.find([
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
544 'ts:(hello <4> world) | (hello<2>world)']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
545 [('test', '3', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
546 ('test', '4', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
547 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
548
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
549 # now check the phrase with near explicitly 3 intervening item
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
550 # with prefix for world.
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
551 self.assertSeqEqual(self.dex.find(['ts:hello <4> wor:*']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
552 [('test', '4', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
553 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
554
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
555 def test_invalid_language(self):
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
556 import psycopg2
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
557
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
558 from roundup.configuration import IndexerOption
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
559 IndexerOption.valid_langs.append("foo")
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
560 self.db.config["INDEXER_LANGUAGE"] = "foo"
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
561
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
562 with self.assertRaises(psycopg2.errors.UndefinedObject) as ctx:
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
563 # psycopg2.errors.UndefinedObject: text search configuration
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
564 # "foo" does not exist
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
565 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
566 self.assertIn('search configuration "foo" does', ctx.exception.args[0])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
567 self.db.rollback()
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
568
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
569 with self.assertRaises(ValueError) as ctx:
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
570 self.dex.find(['"hello" "world"'])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
571 self.assertIn('search configuration "foo" does', ctx.exception.args[0])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
572 self.db.rollback()
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
573
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
574 self.db.config["INDEXER_LANGUAGE"] = "english"
5033
63c79c0992ae Update tests to work with py.test
John Kristensen <john@jerrykan.com>
parents: 4841
diff changeset
575
6915
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
576 def testNullChar(self):
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
577 """Test with null char in string. Postgres FTS throws a ValueError
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
578 on indexing which we ignore. This could happen when
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
579 indexing a binary file with a bad mime type. On find, it
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
580 throws a ProgrammingError that we remap to
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
581 IndexerQueryError and pass up. If a null gets to that
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
582 level on search somebody entered it (not sure how you
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
583 could actually do that) but we want a crash in that case
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
584 as the person is probably up to "no good" (R) (TM).
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
585
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
586 """
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
587 import psycopg2
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
588
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
589 string="\x00\x01fred\x255"
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
590 self.dex.add_text(('test', '1', 'a'), string)
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
591 with self.assertRaises(IndexerQueryError) as ctx:
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
592 self.assertSeqEqual(self.dex.find(string), [])
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
593
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
594 self.assertIn("null", ctx.exception.args[0])
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
595
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
596 @skip_mysql
5033
63c79c0992ae Update tests to work with py.test
John Kristensen <john@jerrykan.com>
parents: 4841
diff changeset
597 class mysqlIndexerTest(mysqlOpener, RDBMSIndexerTest, IndexerTest):
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
598 def setUp(self):
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
599 mysqlOpener.setUp(self)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
600 RDBMSIndexerTest.setUp(self)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
601 def tearDown(self):
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
602 RDBMSIndexerTest.tearDown(self)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
603 mysqlOpener.tearDown(self)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
604
5033
63c79c0992ae Update tests to work with py.test
John Kristensen <john@jerrykan.com>
parents: 4841
diff changeset
605
63c79c0992ae Update tests to work with py.test
John Kristensen <john@jerrykan.com>
parents: 4841
diff changeset
606 class sqliteIndexerTest(sqliteOpener, RDBMSIndexerTest, IndexerTest):
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
607 pass
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
608
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
609 class sqliteFtsIndexerTest(sqliteOpener, RDBMSIndexerTest, IndexerTest):
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
610
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
611 indexer_name = "native-fts"
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
612
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
613 def setUp(self):
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
614 RDBMSIndexerTest.setUp(self)
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
615 from roundup.backends.indexer_sqlite_fts import Indexer
6604
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
616 self.dex = Indexer(self.db)
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
617 self.dex.db = self.db
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
618
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
619 def test_phrase_and_near(self):
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
620 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
621 self.dex.add_text(('test', '2', 'foo'), 'helh blah blah the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
622 self.dex.add_text(('test', '3', 'foo'), 'blah hello the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
623 self.dex.add_text(('test', '4', 'foo'), 'hello blah blech the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
624
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
625 # test two separate words for sanity
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
626 self.assertSeqEqual(self.dex.find(['"hello" "world"']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
627 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
628 ('test', '3', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
629 ('test', '4', 'foo')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
630 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
631 # now check the phrase
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
632 self.assertSeqEqual(self.dex.find(['"hello world"']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
633 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
634 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
635
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
636 # now check the phrase with near explicitly 0 intervening items
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
637 self.assertSeqEqual(self.dex.find(['NEAR(hello world, 0)']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
638 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
639 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
640
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
641 # now check the phrase with near explicitly 1 intervening item
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
642 self.assertSeqEqual(self.dex.find(['NEAR(hello world, 1)']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
643 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
644 ('test', '3', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
645 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
646 # now check the phrase with near explicitly 3 intervening item
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
647 self.assertSeqEqual(self.dex.find(['NEAR(hello world, 3)']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
648 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
649 ('test', '3', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
650 ('test', '4', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
651 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
652
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
653 def test_prefix(self):
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
654 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
655 self.dex.add_text(('test', '2', 'foo'), 'helh blah blah the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
656 self.dex.add_text(('test', '3', 'foo'), 'blah hello the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
657 self.dex.add_text(('test', '4', 'foo'), 'hello blah blech the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
658
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
659 self.assertSeqEqual(self.dex.find(['hel*']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
660 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
661 ('test', '2', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
662 ('test', '3', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
663 ('test', '4', 'foo')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
664 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
665
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
666
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
667 def test_bool_start(self):
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
668 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
669 self.dex.add_text(('test', '2', 'foo'), 'helh blah blah the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
670 self.dex.add_text(('test', '3', 'foo'), 'blah hello the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
671 self.dex.add_text(('test', '4', 'foo'), 'hello blah blech the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
672
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
673 self.assertSeqEqual(self.dex.find(['hel* NOT helh NOT blech']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
674 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
675 ('test', '3', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
676 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
677
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
678 self.assertSeqEqual(self.dex.find(['hel* NOT helh NOT blech OR the']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
679 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
680 ('test', '2', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
681 ('test', '3', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
682 ('test', '4', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
683 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
684
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
685 self.assertSeqEqual(self.dex.find(['helh OR hello']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
686 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
687 ('test', '2', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
688 ('test', '3', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
689 ('test', '4', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
690 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
691
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
692
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
693 self.assertSeqEqual(self.dex.find(['helh AND hello']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
694 [])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
695 # matches if line starts with hello
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
696 self.assertSeqEqual(self.dex.find(['^hello']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
697 [
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
698 ('test', '4', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
699 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
700
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
701 self.assertSeqEqual(self.dex.find(['hello']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
702 [
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
703 ('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
704 ('test', '3', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
705 ('test', '4', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
706 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
707
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
708 def test_query_errors(self):
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
709 """test query phrases that generate an error. Also test the
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
710 correction"""
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
711
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
712 self.dex.add_text(('test', '1', 'foo'), 'a the hello-world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
713 self.dex.add_text(('test', '2', 'foo'), 'helh blah blah the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
714 self.dex.add_text(('test', '3', 'foo'), 'blah hello the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
715 self.dex.add_text(('test', '4', 'foo'), 'hello blah blech the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
716
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
717 # handle known error that roundup recognizes and tries to diagnose
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
718 with self.assertRaises(IndexerQueryError) as ctx:
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
719 self.dex.find(['the hello-world'])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
720
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
721 error = ( "Search failed. Try quoting any terms that include a '-' "
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
722 "and retry the search.")
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
723 self.assertEqual(str(ctx.exception), error)
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
724
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
725
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
726 self.assertSeqEqual(self.dex.find(['the "hello-world"']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
727 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
728 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
729
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
730 # handle known error that roundup recognizes and tries to diagnose
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
731 with self.assertRaises(IndexerQueryError) as ctx:
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
732 self.dex.find(['hello world + ^the'])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
733
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
734 error = 'Query error: syntax error near "^"'
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
735 self.assertEqual(str(ctx.exception), error)
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
736
6915
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
737 def testNullChar(self):
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
738 """Test with null char in string. FTS will throw
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
739 an error on null.
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
740 """
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
741 string="\x00\x01fred\x255"
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
742 self.dex.add_text(('test', '1', 'a'), string)
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
743 with self.assertRaises(IndexerQueryError) as cm:
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
744 self.assertSeqEqual(self.dex.find(string), [])
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
745
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
746 # vim: set filetype=python ts=4 sw=4 et si
Roundup Issue Tracker: http://roundup-tracker.org/