Mercurial > p > roundup > code

annotate test/test_indexer.py @ 7590:5fbd3af526bd

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
fix: issue2551278 - datetime.datetime.utcnow deprecation. We now use the timezone aware utc dates for python 3.11+. But we have to make all the rest of the dates (datetime.min, unix epoch date) timezon aware so we can subtract them. Also need to marshall/unmarshall timezone aware iso formatted date strings.
author John Rouillard <rouilj@ieee.org>
date Thu, 27 Jul 2023 00:53:36 -0400
parents 67e7225c4343
children d17e57220a62
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1 # Copyright (c) 2002 ekit.com Inc (http://www.ekit-inc.com/)
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2 #
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3 # Permission is hereby granted, free of charge, to any person obtaining a copy
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
4 # of this software and associated documentation files (the "Software"), to deal
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
5 # in the Software without restriction, including without limitation the rights
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
6 # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
7 # copies of the Software, and to permit persons to whom the Software is
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
8 # furnished to do so, subject to the following conditions:
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
9 #
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
10 # The above copyright notice and this permission notice shall be included in
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
11 # all copies or substantial portions of the Software.
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
12 #
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
13 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
14 # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
15 # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
16 # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
17 # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
18 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
19 # SOFTWARE.
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
20
7070
c6b2534a58a9 Compatibility: unittest.mock vs. only mock
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6915
diff changeset
21 import os, sys, unittest, shutil
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
22
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
23 import pytest
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
24 from roundup.backends import get_backend, have_backend
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
25 from roundup.backends.indexer_rdbms import Indexer
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
26 from roundup.backends.indexer_common import get_indexer
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
27
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
28 from roundup.cgi.exceptions import IndexerQueryError
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
29
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
30 # borrow from other tests
5388
d26921b851c3 Python 3 preparation: make relative imports explicit.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5109
diff changeset
31 from .db_test_base import setupSchema, config
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
32 from .test_postgresql import postgresqlOpener, skip_postgresql
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
33 from .test_mysql import mysqlOpener, skip_mysql
5388
d26921b851c3 Python 3 preparation: make relative imports explicit.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5109
diff changeset
34 from .test_sqlite import sqliteOpener
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
35 from .test_anydbm import anydbmOpener
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
36
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
37 try:
7070
c6b2534a58a9 Compatibility: unittest.mock vs. only mock
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6915
diff changeset
38 from unittest import mock
c6b2534a58a9 Compatibility: unittest.mock vs. only mock
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6915
diff changeset
39 except ImportError:
c6b2534a58a9 Compatibility: unittest.mock vs. only mock
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6915
diff changeset
40 import mock
c6b2534a58a9 Compatibility: unittest.mock vs. only mock
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6915
diff changeset
41
c6b2534a58a9 Compatibility: unittest.mock vs. only mock
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6915
diff changeset
42 try:
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
43 import xapian
5038
c977f3530944 Work-around for pytest.mark.skipif() bug
John Kristensen <john@jerrykan.com>
parents: 5037
diff changeset
44 skip_xapian = lambda func, *args, **kwargs: func
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
45 except ImportError:
5109
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
46 # FIX: workaround for a bug in pytest.mark.skip():
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
47 # https://github.com/pytest-dev/pytest/issues/568
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
48 from .pytest_patcher import mark_class
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
49 skip_xapian = mark_class(pytest.mark.skip(
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
50 "Skipping Xapian indexer tests: 'xapian' not installed"))
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
51
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
52 try:
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
53 import whoosh
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
54 skip_whoosh = lambda func, *args, **kwargs: func
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
55 except ImportError:
5109
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
56 # FIX: workaround for a bug in pytest.mark.skip():
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
57 # https://github.com/pytest-dev/pytest/issues/568
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
58 from .pytest_patcher import mark_class
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
59 skip_whoosh = mark_class(pytest.mark.skip(
43a1f7fe39f5 Improved work-around for pytest markers bug
John Kristensen <john@jerrykan.com>
parents: 5105
diff changeset
60 "Skipping Whoosh indexer tests: 'whoosh' not installed"))
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
61
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
62 class db:
3546
a4edd24c32be test fixes and checking of indexer overwrites (xapian currently fails)
Richard Jones <richard@users.sourceforge.net>
parents: 3297
diff changeset
63 class config(dict):
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
64 DATABASE = 'test-index'
3546
a4edd24c32be test fixes and checking of indexer overwrites (xapian currently fails)
Richard Jones <richard@users.sourceforge.net>
parents: 3297
diff changeset
65 config = config()
a4edd24c32be test fixes and checking of indexer overwrites (xapian currently fails)
Richard Jones <richard@users.sourceforge.net>
parents: 3297
diff changeset
66 config[('main', 'indexer_stopwords')] = []
6353
9d209d2b34ae Add indexer_language to change stemmer for xapian FTS indexer
John Rouillard <rouilj@ieee.org>
parents: 5964
diff changeset
67 config[('main', 'indexer_language')] = "english"
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
68
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
69 class IndexerTest(anydbmOpener, unittest.TestCase):
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
70
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
71 indexer_name = "native"
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
72
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
73 def setUp(self):
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
74 # remove previous test, ignore errors
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
75 if os.path.exists(config.DATABASE):
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
76 shutil.rmtree(config.DATABASE)
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
77 self.db = self.module.Database(config, 'admin')
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
78
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
79 if os.path.exists('test-index'):
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
80 shutil.rmtree('test-index')
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
81 os.mkdir('test-index')
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
82 os.mkdir('test-index/files')
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
83 from roundup.backends.indexer_dbm import Indexer
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
84 self.dex = Indexer(db)
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
85 self.dex.load_index()
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
86
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
87 def assertSeqEqual(self, s1, s2):
4102
dcca66d56815 fix unit test compatibility
Richard Jones <richard@users.sourceforge.net>
parents: 4016
diff changeset
88 # First argument is the db result we're testing, second is the
dcca66d56815 fix unit test compatibility
Richard Jones <richard@users.sourceforge.net>
parents: 4016
diff changeset
89 # desired result. Some db results don't have iterable rows, so we
dcca66d56815 fix unit test compatibility
Richard Jones <richard@users.sourceforge.net>
parents: 4016
diff changeset
90 # have to work around that.
4015
6eec11b197aa fix for indexer-test:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4008
diff changeset
91 # Also work around some dbs not returning items in the expected
4102
dcca66d56815 fix unit test compatibility
Richard Jones <richard@users.sourceforge.net>
parents: 4016
diff changeset
92 # order.
dcca66d56815 fix unit test compatibility
Richard Jones <richard@users.sourceforge.net>
parents: 4016
diff changeset
93 s1 = list([tuple([r[n] for n in range(len(r))]) for r in s1])
4015
6eec11b197aa fix for indexer-test:
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4008
diff changeset
94 s1.sort()
4102
dcca66d56815 fix unit test compatibility
Richard Jones <richard@users.sourceforge.net>
parents: 4016
diff changeset
95 if s1 != s2:
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
96 self.fail('contents of %r != %r'%(s1, s2))
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
97
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
98 def test_basics(self):
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
99 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
100 self.dex.add_text(('test', '2', 'foo'), 'blah blah the world')
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
101 self.assertSeqEqual(self.dex.find(['world']), [('test', '1', 'foo'),
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
102 ('test', '2', 'foo')])
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
103 self.assertSeqEqual(self.dex.find(['blah']), [('test', '2', 'foo')])
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
104 self.assertSeqEqual(self.dex.find(['blah', 'hello']), [])
6915
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
105 self.assertSeqEqual(self.dex.find([]), [])
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
106
3547
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
107 def test_change(self):
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
108 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
109 self.dex.add_text(('test', '2', 'foo'), 'blah blah the world')
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
110 self.assertSeqEqual(self.dex.find(['world']), [('test', '1', 'foo'),
3547
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
111 ('test', '2', 'foo')])
3546
a4edd24c32be test fixes and checking of indexer overwrites (xapian currently fails)
Richard Jones <richard@users.sourceforge.net>
parents: 3297
diff changeset
112 self.dex.add_text(('test', '1', 'foo'), 'a the hello')
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
113 self.assertSeqEqual(self.dex.find(['world']), [('test', '2', 'foo')])
3546
a4edd24c32be test fixes and checking of indexer overwrites (xapian currently fails)
Richard Jones <richard@users.sourceforge.net>
parents: 3297
diff changeset
114
3547
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
115 def test_clear(self):
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
116 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
117 self.dex.add_text(('test', '2', 'foo'), 'blah blah the world')
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
118 self.assertSeqEqual(self.dex.find(['world']), [('test', '1', 'foo'),
3547
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
119 ('test', '2', 'foo')])
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
120 self.dex.add_text(('test', '1', 'foo'), '')
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
121 self.assertSeqEqual(self.dex.find(['world']), [('test', '2', 'foo')])
3547
7728ee93efd2 fix reindexing in Xapian
Richard Jones <richard@users.sourceforge.net>
parents: 3546
diff changeset
122
6912
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
123 def test_get_indexer_specified(self):
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
124 """Specify an indexer back end and make sure it's returned"""
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
125 def class_name_of(object):
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
126 """ take and object and return just the class name.
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
127 So in:
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
128
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
129 return the class name before "at".
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
130
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
131 """
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
132 return(str(object).split()[0])
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
133
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
134 old_indexer = self.db.config['INDEXER']
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
135 self.db.config['INDEXER'] = self.indexer_name
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
136
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
137 self.assertEqual(class_name_of(self.dex),
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
138 class_name_of(get_indexer(self.db.config, self.db)))
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
139
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
140 self.db.config['INDEXER'] = old_indexer
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
141
4251
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
142 def test_stopwords(self):
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
143 """Test that we can find a text with a stopword in it."""
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
144 stopword = "with"
5649
f8893e1cde0d assert_ is depricated. Replacing with assertTrue to reduce logs in travisci.
John Rouillard <rouilj@ieee.org>
parents: 5388
diff changeset
145 self.assertTrue(self.dex.is_stopword(stopword.upper()))
4251
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
146 self.dex.add_text(('test', '1', 'bar'), '%s hello world' % stopword)
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
147 self.dex.add_text(('test', '2', 'bar'), 'blah a %s world' % stopword)
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
148 self.dex.add_text(('test', '3', 'bar'), 'blah Blub river')
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
149 self.dex.add_text(('test', '4', 'bar'), 'blah river %s' % stopword)
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
150 self.assertSeqEqual(self.dex.find(['with','world']),
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
151 [('test', '1', 'bar'),
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
152 ('test', '2', 'bar')])
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
153 def test_extremewords(self):
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
154 """Testing too short or too long words."""
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
155
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
156 # skip this for FTS test
6604
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
157 if ( isinstance(self,sqliteFtsIndexerTest) or
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
158 isinstance(self,postgresqlFtsIndexerTest)):
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
159 pytest.skip("extremewords not tested for native FTS backends")
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
160
4251
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
161 short = "b"
6599
39189dd94f2c issue2551189 - increase size of words in full text index.
John Rouillard <rouilj@ieee.org>
parents: 6588
diff changeset
162 long = "abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz"
4251
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
163 self.dex.add_text(('test', '1', 'a'), '%s hello world' % short)
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
164 self.dex.add_text(('test', '2', 'a'), 'blah a %s world' % short)
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
165 self.dex.add_text(('test', '3', 'a'), 'blah Blub river')
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
166 self.dex.add_text(('test', '4', 'a'), 'blah river %s %s'
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
167 % (short, long))
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
168 self.assertSeqEqual(self.dex.find([short,'world', long, short]),
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
169 [('test', '1', 'a'),
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
170 ('test', '2', 'a')])
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
171 self.assertSeqEqual(self.dex.find([long]),[])
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
172
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
173 # special test because some faulty code indexed length(word)>=2
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
174 # but only considered length(word)>=3 to be significant
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
175 self.dex.add_text(('test', '5', 'a'), 'blah py %s %s'
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
176 % (short, long))
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
177 self.assertSeqEqual(self.dex.find(["py"]), [('test', '5', 'a')])
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
178
6604
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
179 def test_casesensitivity(self):
4251
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
180 """Test if searches are case-in-sensitive."""
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
181 self.dex.add_text(('test', '1', 'a'), 'aaaa bbbb')
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
182 self.dex.add_text(('test', '2', 'a'), 'aAaa BBBB')
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
183 self.assertSeqEqual(self.dex.find(['aaaa']),
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
184 [('test', '1', 'a'),
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
185 ('test', '2', 'a')])
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
186 self.assertSeqEqual(self.dex.find(['BBBB']),
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
187 [('test', '1', 'a'),
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
188 ('test', '2', 'a')])
2b1241daaa20 Added more indexer tests for stopwords, case-insensitity...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4102
diff changeset
189
4314
b41a033bffcc - add a small word-splitting test for the indexers...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4251
diff changeset
190 def test_wordsplitting(self):
b41a033bffcc - add a small word-splitting test for the indexers...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4251
diff changeset
191 """Test if word splitting works."""
b41a033bffcc - add a small word-splitting test for the indexers...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4251
diff changeset
192 self.dex.add_text(('test', '1', 'a'), 'aaaa-aaa bbbb*bbb')
b41a033bffcc - add a small word-splitting test for the indexers...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4251
diff changeset
193 self.dex.add_text(('test', '2', 'a'), 'aaaA-aaa BBBB*BBB')
b41a033bffcc - add a small word-splitting test for the indexers...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4251
diff changeset
194 for k in 'aaaa', 'aaa', 'bbbb', 'bbb':
b41a033bffcc - add a small word-splitting test for the indexers...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4251
diff changeset
195 self.assertSeqEqual(self.dex.find([k]),
b41a033bffcc - add a small word-splitting test for the indexers...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4251
diff changeset
196 [('test', '1', 'a'), ('test', '2', 'a')])
b41a033bffcc - add a small word-splitting test for the indexers...
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 4251
diff changeset
197
4841
3ff1a288fb9c issue2550583, issue2550635 Do not limit results with Xapian indexer
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4570
diff changeset
198 def test_manyresults(self):
3ff1a288fb9c issue2550583, issue2550635 Do not limit results with Xapian indexer
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4570
diff changeset
199 """Test if searches find many results."""
3ff1a288fb9c issue2550583, issue2550635 Do not limit results with Xapian indexer
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4570
diff changeset
200 for i in range(123):
3ff1a288fb9c issue2550583, issue2550635 Do not limit results with Xapian indexer
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4570
diff changeset
201 self.dex.add_text(('test', str(i), 'many'), 'many')
3ff1a288fb9c issue2550583, issue2550635 Do not limit results with Xapian indexer
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4570
diff changeset
202 self.assertEqual(len(self.dex.find(['many'])), 123)
3ff1a288fb9c issue2550583, issue2550635 Do not limit results with Xapian indexer
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4570
diff changeset
203
5960
0db2621b6fee Add test for issue1344046 and maybe issue1195739
John Rouillard <rouilj@ieee.org>
parents: 5649
diff changeset
204 def test_unicode(self):
0db2621b6fee Add test for issue1344046 and maybe issue1195739
John Rouillard <rouilj@ieee.org>
parents: 5649
diff changeset
205 """Test with unicode words. see:
0db2621b6fee Add test for issue1344046 and maybe issue1195739
John Rouillard <rouilj@ieee.org>
parents: 5649
diff changeset
206 https://issues.roundup-tracker.org/issue1344046"""
5962
6137ea845438 Fix russian search/string
John Rouillard <rouilj@ieee.org>
parents: 5961
diff changeset
207 russian=u'\u0440\u0443\u0441\u0441\u043a\u0438\u0439 \u0442\u0435\u043a\u0441\u0442Spr\xfcnge'
5961
f6c58a7b535c Add a russian string for unicode testing of indexer
John Rouillard <rouilj@ieee.org>
parents: 5960
diff changeset
208 german=u'Spr\xfcnge'
f6c58a7b535c Add a russian string for unicode testing of indexer
John Rouillard <rouilj@ieee.org>
parents: 5960
diff changeset
209 self.dex.add_text(('test', '1', 'a'), german )
f6c58a7b535c Add a russian string for unicode testing of indexer
John Rouillard <rouilj@ieee.org>
parents: 5960
diff changeset
210 self.dex.add_text(('test', '2', 'a'), russian + u' ' + german )
f6c58a7b535c Add a russian string for unicode testing of indexer
John Rouillard <rouilj@ieee.org>
parents: 5960
diff changeset
211
f6c58a7b535c Add a russian string for unicode testing of indexer
John Rouillard <rouilj@ieee.org>
parents: 5960
diff changeset
212 self.assertSeqEqual(self.dex.find([ u'Spr\xfcnge']),
f6c58a7b535c Add a russian string for unicode testing of indexer
John Rouillard <rouilj@ieee.org>
parents: 5960
diff changeset
213 [('test', '1', 'a'), ('test', '2', 'a')])
5962
6137ea845438 Fix russian search/string
John Rouillard <rouilj@ieee.org>
parents: 5961
diff changeset
214 self.assertSeqEqual(self.dex.find([u'\u0440\u0443\u0441\u0441\u043a\u0438\u0439']),
5961
f6c58a7b535c Add a russian string for unicode testing of indexer
John Rouillard <rouilj@ieee.org>
parents: 5960
diff changeset
215 [('test', '2', 'a')])
6915
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
216
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
217 def testNullChar(self):
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
218 """Test with null char in string. Postgres FTS will not index
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
219 it will just ignore string for now.
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
220 """
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
221 string="\x00\x01fred\x255"
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
222 self.dex.add_text(('test', '1', 'a'), string)
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
223 self.assertSeqEqual(self.dex.find(string), [])
5960
0db2621b6fee Add test for issue1344046 and maybe issue1195739
John Rouillard <rouilj@ieee.org>
parents: 5649
diff changeset
224
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
225 def tearDown(self):
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
226 shutil.rmtree('test-index')
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
227 if hasattr(self, 'db'):
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
228 self.db.close()
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
229 if os.path.exists(config.DATABASE):
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
230 shutil.rmtree(config.DATABASE)
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
231
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
232 @skip_whoosh
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
233 class WhooshIndexerTest(IndexerTest):
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
234
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
235 indexer_name = "whoosh"
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
236
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
237 def setUp(self):
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
238 IndexerTest.setUp(self)
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
239
5096
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
240 if os.path.exists('test-index'):
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
241 shutil.rmtree('test-index')
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
242 os.mkdir('test-index')
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
243 from roundup.backends.indexer_whoosh import Indexer
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
244 self.dex = Indexer(db)
e74c3611b138 - issue2550636, issue2550909: Added support for Whoosh indexer.
John Rouillard <rouilj@ieee.org>
parents: 5038
diff changeset
245 def tearDown(self):
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
246 IndexerTest.tearDown(self)
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
247
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
248 @skip_xapian
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
249 class XapianIndexerTest(IndexerTest):
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
250
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
251 indexer_name = "xapian"
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
252
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
253 def setUp(self):
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
254 IndexerTest.setUp(self)
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
255
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
256 if os.path.exists('test-index'):
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
257 shutil.rmtree('test-index')
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
258 os.mkdir('test-index')
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
259 from roundup.backends.indexer_xapian import Indexer
3297
8f7dc283bfa5 some more Xapian stuff (doc, test fixes)
Richard Jones <richard@users.sourceforge.net>
parents: 3295
diff changeset
260 self.dex = Indexer(db)
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3078
diff changeset
261 def tearDown(self):
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
262 IndexerTest.tearDown(self)
5033
63c79c0992ae Update tests to work with py.test
John Kristensen <john@jerrykan.com>
parents: 4841
diff changeset
263
6915
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
264 class Get_IndexerTest(anydbmOpener, unittest.TestCase):
6912
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
265
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
266 def setUp(self):
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
267 # remove previous test, ignore errors
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
268 if os.path.exists(config.DATABASE):
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
269 shutil.rmtree(config.DATABASE)
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
270 self.db = self.module.Database(config, 'admin')
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
271 # this is the default, but set it in case default
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
272 # changes in future
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
273 self.db.config['INDEXER'] = ''
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
274
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
275 def tearDown(self):
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
276 if hasattr(self, 'db'):
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
277 self.db.close()
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
278 if os.path.exists(config.DATABASE):
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
279 shutil.rmtree(config.DATABASE)
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
280
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
281 @skip_xapian
6915
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
282 def test_xapian_autoselect(self):
6912
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
283 indexer = get_indexer(self.db.config, self.db)
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
284 self.assertIn('roundup.backends.indexer_xapian.Indexer', str(indexer))
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
285
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
286 @skip_whoosh
6915
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
287 def test_whoosh_autoselect(self):
6912
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
288 with mock.patch.dict('sys.modules',
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
289 {'roundup.backends.indexer_xapian': None}):
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
290 indexer = get_indexer(self.db.config, self.db)
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
291 self.assertIn('roundup.backends.indexer_whoosh.Indexer', str(indexer))
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
292
6915
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
293 def test_native_autoselect(self):
6912
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
294 with mock.patch.dict('sys.modules',
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
295 {'roundup.backends.indexer_xapian': None,
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
296 'roundup.backends.indexer_whoosh': None}):
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
297 indexer = get_indexer(self.db.config, self.db)
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
298 self.assertIn('roundup.backends.indexer_dbm.Indexer', str(indexer))
a23eaa3013e6 test get_indexer three autosearch options: xapian, whoosh, native
John Rouillard <rouilj@ieee.org>
parents: 6909
diff changeset
299
6915
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
300 def test_invalid_indexer(self):
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
301 """There is code at the end of indexer_common::get_indexer() to
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
302 raise an AssertionError if the indexer name is invalid.
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
303 This should never be triggered. If it is, it means that
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
304 the code in configure.py that validates indexer names
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
305 allows a name through that get_indexer can't handle.
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
306
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
307 Simulate that failure and make sure that the
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
308 AssertionError is raised.
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
309
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
310 """
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
311
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
312 with self.assertRaises(ValueError) as cm:
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
313 self.db.config['INDEXER'] = 'no_such_indexer'
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
314
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
315 # mangle things so we can test AssertionError at end
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
316 # get_indexer()
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
317 from roundup.configuration import IndexerOption
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
318 IndexerOption.allowed.append("unrecognized_indexer")
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
319 self.db.config['INDEXER'] = "unrecognized_indexer"
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
320
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
321 with self.assertRaises(AssertionError) as cm:
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
322 indexer = get_indexer(self.db.config, self.db)
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
323
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
324 # unmangle state
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
325 IndexerOption.allowed.pop()
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
326 self.assertNotIn("unrecognized_indexer", IndexerOption.allowed)
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
327 self.db.config['INDEXER'] = ""
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
328
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
329 def test_unsupported_by_db(self):
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
330 """This requires that the db associated with the test
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
331 is not sqlite or postgres. anydbm works fine to trigger
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
332 the error.
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
333 """
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
334 self.db.config['INDEXER'] = 'native-fts'
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
335 with self.assertRaises(AssertionError) as cm:
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
336 get_indexer(self.db.config, self.db)
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
337
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
338 self.assertIn("native-fts", cm.exception.args[0])
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
339 self.db.config['INDEXER'] = ''
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
340
5033
63c79c0992ae Update tests to work with py.test
John Kristensen <john@jerrykan.com>
parents: 4841
diff changeset
341 class RDBMSIndexerTest(object):
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
342 def setUp(self):
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
343 # remove previous test, ignore errors
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
344 if os.path.exists(config.DATABASE):
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
345 shutil.rmtree(config.DATABASE)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
346 self.db = self.module.Database(config, 'admin')
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
347 self.dex = Indexer(self.db)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
348 def tearDown(self):
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
349 if hasattr(self, 'db'):
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
350 self.db.close()
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
351 if os.path.exists(config.DATABASE):
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
352 shutil.rmtree(config.DATABASE)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
353
5033
63c79c0992ae Update tests to work with py.test
John Kristensen <john@jerrykan.com>
parents: 4841
diff changeset
354
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
355 @skip_postgresql
5033
63c79c0992ae Update tests to work with py.test
John Kristensen <john@jerrykan.com>
parents: 4841
diff changeset
356 class postgresqlIndexerTest(postgresqlOpener, RDBMSIndexerTest, IndexerTest):
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
357 def setUp(self):
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
358 postgresqlOpener.setUp(self)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
359 RDBMSIndexerTest.setUp(self)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
360 def tearDown(self):
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
361 RDBMSIndexerTest.tearDown(self)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
362 postgresqlOpener.tearDown(self)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
363
6604
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
364
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
365 @skip_postgresql
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
366 class postgresqlFtsIndexerTest(postgresqlOpener, RDBMSIndexerTest, IndexerTest):
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
367
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
368 indexer_name = "native-fts"
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
369
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
370 def setUp(self):
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
371 postgresqlOpener.setUp(self)
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
372 RDBMSIndexerTest.setUp(self)
6604
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
373 from roundup.backends.indexer_postgresql_fts import Indexer
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
374 self.dex = Indexer(self.db)
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
375 self.dex.db = self.db
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
376
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
377 def tearDown(self):
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
378 RDBMSIndexerTest.tearDown(self)
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
379 postgresqlOpener.tearDown(self)
6604
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
380
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
381 def test_websearch_syntax(self):
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
382 """Test searches using websearch_to_tsquery. These never throw
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
383 errors regardless of how wacky the input.
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
384 """
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
385
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
386 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
387 self.dex.add_text(('test', '2', 'foo'), 'helh blah blah the world')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
388 self.dex.add_text(('test', '3', 'foo'), 'blah hello the world')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
389 self.dex.add_text(('test', '4', 'foo'), 'hello blah blech the world')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
390 self.dex.add_text(('test', '5', 'foo'), 'a car drove')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
391 self.dex.add_text(('test', '6', 'foo'), 'a car driving itself')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
392 self.dex.add_text(('test', '7', 'foo'), "let's drive in the car")
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
393 self.dex.add_text(('test', '8', 'foo'), 'a drive-in movie')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
394
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
395 # test two separate words for sanity
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
396 self.assertSeqEqual(self.dex.find(['"hello" "world"']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
397 [('test', '1', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
398 ('test', '3', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
399 ('test', '4', 'foo')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
400 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
401 # now check the phrase
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
402 self.assertSeqEqual(self.dex.find(['"hello world"']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
403 [('test', '1', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
404 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
405
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
406 # test negation
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
407 self.assertSeqEqual(self.dex.find(['hello world -blech']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
408 [('test', '1', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
409 ('test', '3', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
410 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
411
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
412 # phrase negation
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
413 self.assertSeqEqual(self.dex.find(['hello world -"blah hello"']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
414 [('test', '1', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
415 ('test', '4', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
416 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
417
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
418 # test without or
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
419 self.assertSeqEqual(self.dex.find(['blah blech']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
420 [('test', '4', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
421 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
422
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
423 # test with or
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
424 self.assertSeqEqual(self.dex.find(['blah or blech']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
425 [ ('test', '2', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
426 ('test', '3', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
427 ('test', '4', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
428 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
429
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
430 # stemmer test for english
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
431 self.assertSeqEqual(self.dex.find(['ts:drive']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
432 [('test', '6', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
433 ('test', '7', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
434 ('test', '8', 'foo')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
435 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
436
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
437 # stemmer is not disabled by quotes 8-(
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
438 self.assertSeqEqual(self.dex.find(['ts:"drive"']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
439 [('test', '6', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
440 ('test', '7', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
441 ('test', '8', 'foo')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
442 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
443
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
444
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
445 # this is missing ts: at the start, so uses the websearch
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
446 # parser. We search for operator characters and wanr the user
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
447 # Otherwise "hello <-> world" is the same as "hello world"
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
448 # and is not a phrase search.
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
449 with self.assertRaises(IndexerQueryError) as ctx:
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
450 self.dex.find(['hello <-> world'])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
451
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
452 self.assertIn('do a tsquery search', ctx.exception.args[0])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
453
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
454 def test_tsquery_syntax(self):
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
455 """Because websearch_to_tsquery doesn't allow prefix searches,
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
456 near searches with any value except 1 (phrase search), allow
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
457 use of to_tsquery by prefixing the search term wih ts:.
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
458
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
459 However, unlike websearch_to_tsquery, this will throw a
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
460 psycopg2.errors.SyntaxError on bad input. SyntaxError is
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
461 re-raised as IndexerQueryError. But it makes a bunch of
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
462 useful expert functionality available.
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
463
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
464 """
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
465
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
466 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
467 self.dex.add_text(('test', '2', 'foo'), 'helh blah blah the world')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
468 self.dex.add_text(('test', '3', 'foo'), 'blah hello the world')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
469 self.dex.add_text(('test', '4', 'foo'), 'hello blah blech the world')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
470 self.dex.add_text(('test', '5', 'foo'), 'a car drove')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
471 self.dex.add_text(('test', '6', 'foo'), 'a car driving itself')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
472 self.dex.add_text(('test', '7', 'foo'), "let's drive in the car")
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
473 self.dex.add_text(('test', '8', 'foo'), 'a drive-in movie')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
474 self.dex.db.commit()
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
475
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
476 # test two separate words for sanity
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
477 self.assertSeqEqual(self.dex.find(['ts:hello & world']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
478 [('test', '1', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
479 ('test', '3', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
480 ('test', '4', 'foo')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
481 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
482 # now check the phrase
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
483 self.assertSeqEqual(self.dex.find(['ts:hello <-> world']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
484 [('test', '1', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
485 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
486
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
487 # test negation
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
488 self.assertSeqEqual(self.dex.find(['ts:hello & world & !blech']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
489 [('test', '1', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
490 ('test', '3', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
491 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
492
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
493 self.assertSeqEqual(self.dex.find(
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
494 ['ts:hello & world & !(blah <-> hello)']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
495 [('test', '1', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
496 ('test', '4', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
497 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
498
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
499 # test without or
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
500 self.assertSeqEqual(self.dex.find(['ts:blah & blech']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
501 [('test', '4', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
502 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
503
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
504 # test with or
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
505 self.assertSeqEqual(self.dex.find(['ts:blah | blech']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
506 [ ('test', '2', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
507 ('test', '3', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
508 ('test', '4', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
509 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
510 # stemmer test for english
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
511 self.assertSeqEqual(self.dex.find(['ts:drive']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
512 [('test', '6', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
513 ('test', '7', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
514 ('test', '8', 'foo')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
515 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
516
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
517 # stemmer is not disabled by quotes 8-(
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
518 self.assertSeqEqual(self.dex.find(['ts:"drive"']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
519 [('test', '6', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
520 ('test', '7', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
521 ('test', '8', 'foo')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
522 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
523
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
524
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
525 # test with syntax error
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
526 with self.assertRaises(IndexerQueryError) as ctx:
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
527 self.dex.find(['ts:blah blech'])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
528
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
529 self.assertEqual(ctx.exception.args[0],
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
530 'syntax error in tsquery: "blah blech"\n')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
531
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
532 # now check the phrase Note unlike sqlite, order matters,
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
533 # hello must come first.
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
534 self.assertSeqEqual(self.dex.find(['ts:hello <-> world']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
535 [('test', '1', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
536 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
537
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
538 # now check the phrase with explicitly 1 intervening item
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
539 self.assertSeqEqual(self.dex.find(['ts:hello <2> world']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
540 [('test', '3', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
541 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
542 # now check the phrase with near explicitly 1 or 3 intervening items
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
543 self.assertSeqEqual(self.dex.find([
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
544 'ts:(hello <4> world) | (hello<2>world)']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
545 [('test', '3', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
546 ('test', '4', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
547 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
548
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
549 # now check the phrase with near explicitly 3 intervening item
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
550 # with prefix for world.
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
551 self.assertSeqEqual(self.dex.find(['ts:hello <4> wor:*']),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
552 [('test', '4', 'foo'),
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
553 ])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
554
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
555 def test_invalid_language(self):
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
556 import psycopg2
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
557
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
558 from roundup.configuration import IndexerOption
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
559 IndexerOption.valid_langs.append("foo")
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
560 self.db.config["INDEXER_LANGUAGE"] = "foo"
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
561
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
562 with self.assertRaises(psycopg2.errors.UndefinedObject) as ctx:
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
563 # psycopg2.errors.UndefinedObject: text search configuration
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
564 # "foo" does not exist
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
565 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
566 self.assertIn('search configuration "foo" does', ctx.exception.args[0])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
567 self.db.rollback()
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
568
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
569 with self.assertRaises(ValueError) as ctx:
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
570 self.dex.find(['"hello" "world"'])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
571 self.assertIn('search configuration "foo" does', ctx.exception.args[0])
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
572 self.db.rollback()
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
573
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
574 self.db.config["INDEXER_LANGUAGE"] = "english"
5033
63c79c0992ae Update tests to work with py.test
John Kristensen <john@jerrykan.com>
parents: 4841
diff changeset
575
6915
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
576 def testNullChar(self):
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
577 """Test with null char in string. Postgres FTS throws a ValueError
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
578 on indexing which we ignore. This could happen when
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
579 indexing a binary file with a bad mime type. On find, it
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
580 throws a ProgrammingError that we remap to
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
581 IndexerQueryError and pass up. If a null gets to that
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
582 level on search somebody entered it (not sure how you
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
583 could actually do that) but we want a crash in that case
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
584 as the person is probably up to "no good" (R) (TM).
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
585
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
586 """
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
587 import psycopg2
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
588
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
589 string="\x00\x01fred\x255"
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
590 self.dex.add_text(('test', '1', 'a'), string)
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
591 with self.assertRaises(IndexerQueryError) as ctx:
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
592 self.assertSeqEqual(self.dex.find(string), [])
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
593
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
594 self.assertIn("null", ctx.exception.args[0])
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
595
5036
380d8d8b30a3 Replace existing run_tests.py script with a pytest script
John Kristensen <john@jerrykan.com>
parents: 5033
diff changeset
596 @skip_mysql
5033
63c79c0992ae Update tests to work with py.test
John Kristensen <john@jerrykan.com>
parents: 4841
diff changeset
597 class mysqlIndexerTest(mysqlOpener, RDBMSIndexerTest, IndexerTest):
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
598 def setUp(self):
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
599 mysqlOpener.setUp(self)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
600 RDBMSIndexerTest.setUp(self)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
601 def tearDown(self):
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
602 RDBMSIndexerTest.tearDown(self)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
603 mysqlOpener.tearDown(self)
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
604
5033
63c79c0992ae Update tests to work with py.test
John Kristensen <john@jerrykan.com>
parents: 4841
diff changeset
605
63c79c0992ae Update tests to work with py.test
John Kristensen <john@jerrykan.com>
parents: 4841
diff changeset
606 class sqliteIndexerTest(sqliteOpener, RDBMSIndexerTest, IndexerTest):
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
607 pass
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3547
diff changeset
608
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
609 class sqliteFtsIndexerTest(sqliteOpener, RDBMSIndexerTest, IndexerTest):
6909
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
610
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
611 indexer_name = "native-fts"
86cde9cae7e1 Improve test coverage of backends/indexer_common::get_indexer() method
John Rouillard <rouilj@ieee.org>
parents: 6908
diff changeset
612
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
613 def setUp(self):
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
614 RDBMSIndexerTest.setUp(self)
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
615 from roundup.backends.indexer_sqlite_fts import Indexer
6604
0d99ae7c8de6 Allow Roundup to use PostgreSQL database native full text search
John Rouillard <rouilj@ieee.org>
parents: 6599
diff changeset
616 self.dex = Indexer(self.db)
6588
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
617 self.dex.db = self.db
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
618
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
619 def test_phrase_and_near(self):
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
620 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
621 self.dex.add_text(('test', '2', 'foo'), 'helh blah blah the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
622 self.dex.add_text(('test', '3', 'foo'), 'blah hello the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
623 self.dex.add_text(('test', '4', 'foo'), 'hello blah blech the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
624
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
625 # test two separate words for sanity
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
626 self.assertSeqEqual(self.dex.find(['"hello" "world"']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
627 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
628 ('test', '3', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
629 ('test', '4', 'foo')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
630 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
631 # now check the phrase
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
632 self.assertSeqEqual(self.dex.find(['"hello world"']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
633 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
634 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
635
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
636 # now check the phrase with near explicitly 0 intervening items
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
637 self.assertSeqEqual(self.dex.find(['NEAR(hello world, 0)']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
638 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
639 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
640
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
641 # now check the phrase with near explicitly 1 intervening item
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
642 self.assertSeqEqual(self.dex.find(['NEAR(hello world, 1)']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
643 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
644 ('test', '3', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
645 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
646 # now check the phrase with near explicitly 3 intervening item
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
647 self.assertSeqEqual(self.dex.find(['NEAR(hello world, 3)']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
648 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
649 ('test', '3', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
650 ('test', '4', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
651 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
652
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
653 def test_prefix(self):
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
654 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
655 self.dex.add_text(('test', '2', 'foo'), 'helh blah blah the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
656 self.dex.add_text(('test', '3', 'foo'), 'blah hello the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
657 self.dex.add_text(('test', '4', 'foo'), 'hello blah blech the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
658
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
659 self.assertSeqEqual(self.dex.find(['hel*']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
660 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
661 ('test', '2', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
662 ('test', '3', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
663 ('test', '4', 'foo')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
664 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
665
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
666
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
667 def test_bool_start(self):
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
668 self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
669 self.dex.add_text(('test', '2', 'foo'), 'helh blah blah the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
670 self.dex.add_text(('test', '3', 'foo'), 'blah hello the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
671 self.dex.add_text(('test', '4', 'foo'), 'hello blah blech the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
672
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
673 self.assertSeqEqual(self.dex.find(['hel* NOT helh NOT blech']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
674 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
675 ('test', '3', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
676 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
677
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
678 self.assertSeqEqual(self.dex.find(['hel* NOT helh NOT blech OR the']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
679 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
680 ('test', '2', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
681 ('test', '3', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
682 ('test', '4', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
683 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
684
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
685 self.assertSeqEqual(self.dex.find(['helh OR hello']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
686 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
687 ('test', '2', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
688 ('test', '3', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
689 ('test', '4', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
690 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
691
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
692
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
693 self.assertSeqEqual(self.dex.find(['helh AND hello']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
694 [])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
695 # matches if line starts with hello
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
696 self.assertSeqEqual(self.dex.find(['^hello']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
697 [
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
698 ('test', '4', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
699 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
700
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
701 self.assertSeqEqual(self.dex.find(['hello']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
702 [
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
703 ('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
704 ('test', '3', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
705 ('test', '4', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
706 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
707
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
708 def test_query_errors(self):
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
709 """test query phrases that generate an error. Also test the
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
710 correction"""
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
711
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
712 self.dex.add_text(('test', '1', 'foo'), 'a the hello-world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
713 self.dex.add_text(('test', '2', 'foo'), 'helh blah blah the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
714 self.dex.add_text(('test', '3', 'foo'), 'blah hello the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
715 self.dex.add_text(('test', '4', 'foo'), 'hello blah blech the world')
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
716
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
717 # handle known error that roundup recognizes and tries to diagnose
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
718 with self.assertRaises(IndexerQueryError) as ctx:
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
719 self.dex.find(['the hello-world'])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
720
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
721 error = ( "Search failed. Try quoting any terms that include a '-' "
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
722 "and retry the search.")
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
723 self.assertEqual(str(ctx.exception), error)
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
724
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
725
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
726 self.assertSeqEqual(self.dex.find(['the "hello-world"']),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
727 [('test', '1', 'foo'),
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
728 ])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
729
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
730 # handle known error that roundup recognizes and tries to diagnose
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
731 with self.assertRaises(IndexerQueryError) as ctx:
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
732 self.dex.find(['hello world + ^the'])
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
733
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
734 error = 'Query error: syntax error near "^"'
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
735 self.assertEqual(str(ctx.exception), error)
91ab3e0ffcd0 Summary: Add test cases for sqlite fts
John Rouillard <rouilj@ieee.org>
parents: 6356
diff changeset
736
6915
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
737 def testNullChar(self):
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
738 """Test with null char in string. FTS will throw
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
739 an error on null.
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
740 """
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
741 string="\x00\x01fred\x255"
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
742 self.dex.add_text(('test', '1', 'a'), string)
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
743 with self.assertRaises(IndexerQueryError) as cm:
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
744 self.assertSeqEqual(self.dex.find(string), [])
9ff091537f43 postgresql native-fts; more indexer tests
John Rouillard <rouilj@ieee.org>
parents: 6912
diff changeset
745
848
2a928d404af8 ehem, forgot to add
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
746 # vim: set filetype=python ts=4 sw=4 et si
Roundup Issue Tracker: http://roundup-tracker.org/