Mercurial > p > roundup > code

annotate roundup/backends/indexer_rdbms.py @ 4252:2ff6f39aa391

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Indexers behaviour made more consistent regarding length of indexed words... ...and stopwords (thanks Thomas Arendsen Hein, Bernhard Reiter)(issue 2550584)
author Bernhard Reiter <Bernhard.Reiter@intevation.de>
date Fri, 11 Sep 2009 15:55:11 +0000
parents 09e79cbeb827
children 8081d34fefa5
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3988
diff changeset
1 #$Id: indexer_rdbms.py,v 1.18 2008-09-01 00:43:02 richard Exp $
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4008
diff changeset
2 """ This implements the full-text indexer over two RDBMS tables. The first
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3 is a mapping of words to occurance IDs. The second maps the IDs to (Class,
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
4 propname, itemid) instances.
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4008
diff changeset
5 """
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4008
diff changeset
6 import re
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4008
diff changeset
7 # Python 2.3 ... 2.6 compatibility:
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4008
diff changeset
8 from roundup.anypy.sets_ import set
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
9
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3414
diff changeset
10 from roundup.backends.indexer_common import Indexer as IndexerBase
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
11
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3414
diff changeset
12 class Indexer(IndexerBase):
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
13 def __init__(self, db):
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3414
diff changeset
14 IndexerBase.__init__(self, db)
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
15 self.db = db
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
16 self.reindex = 0
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
17
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
18 def close(self):
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4008
diff changeset
19 """close the indexing database"""
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
20 # just nuke the circular reference
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
21 self.db = None
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3092
diff changeset
22
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3092
diff changeset
23 def save_index(self):
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4008
diff changeset
24 """Save the changes to the index."""
3295
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3092
diff changeset
25 # not necessary - the RDBMS connection will handle this for us
a615cc230160 added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents: 3092
diff changeset
26 pass
3331
7bc09d5d9544 perform word splitting in unicode for national characters support
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3295
diff changeset
27
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
28 def force_reindex(self):
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4008
diff changeset
29 """Force a reindexing of the database. This essentially
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
30 empties the tables ids and index and sets a flag so
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4008
diff changeset
31 that the databases are reindexed"""
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
32 self.reindex = 1
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
33
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
34 def should_reindex(self):
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4008
diff changeset
35 """returns True if the indexes need to be rebuilt"""
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
36 return self.reindex
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
37
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
38 def add_text(self, identifier, text, mime_type='text/plain'):
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4008
diff changeset
39 """ "identifier" is (classname, itemid, property) """
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
40 if mime_type != 'text/plain':
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
41 return
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
42
3988
b50446ff16f4 fix comment, thanks alex
Richard Jones <richard@users.sourceforge.net>
parents: 3981
diff changeset
43 # Ensure all elements of the identifier are strings 'cos the itemid
3981
82e116d515d2 another psql 8.3 compat fix
Richard Jones <richard@users.sourceforge.net>
parents: 3718
diff changeset
44 # column is varchar even if item ids may be numbers elsewhere in the
82e116d515d2 another psql 8.3 compat fix
Richard Jones <richard@users.sourceforge.net>
parents: 3718
diff changeset
45 # code. ugh.
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3988
diff changeset
46 identifier = tuple(map(str, identifier))
3981
82e116d515d2 another psql 8.3 compat fix
Richard Jones <richard@users.sourceforge.net>
parents: 3718
diff changeset
47
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
48 # first, find the id of the (classname, itemid, property)
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
49 a = self.db.arg
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
50 sql = 'select _textid from __textids where _class=%s and '\
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
51 '_itemid=%s and _prop=%s'%(a, a, a)
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
52 self.db.cursor.execute(sql, identifier)
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
53 r = self.db.cursor.fetchone()
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
54 if not r:
3617
f12722c7b9ee improvements
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
55 # not previously indexed
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
56 id = self.db.newid('__textids')
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
57 sql = 'insert into __textids (_textid, _class, _itemid, _prop)'\
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
58 ' values (%s, %s, %s, %s)'%(a, a, a, a)
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
59 self.db.cursor.execute(sql, (id, ) + identifier)
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
60 else:
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
61 id = int(r[0])
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
62 # clear out any existing indexed values
2098
18addf2a8596 Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents: 2093
diff changeset
63 sql = 'delete from __words where _textid=%s'%a
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
64 self.db.cursor.execute(sql, (id, ))
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
65
3617
f12722c7b9ee improvements
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
66 # ok, find all the unique words in the text
3331
7bc09d5d9544 perform word splitting in unicode for national characters support
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3295
diff changeset
67 text = unicode(text, "utf-8", "replace").upper()
4091
09e79cbeb827 force sqlite3 in py2.6+ to treat our stored text as UTF-8
Richard Jones <richard@users.sourceforge.net>
parents: 4089
diff changeset
68 wordlist = [w.encode("utf-8")
4252
2ff6f39aa391 Indexers behaviour made more consistent regarding length of indexed words...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4091
diff changeset
69 for w in re.findall(r'(?u)\b\w{%d,%d}\b'
2ff6f39aa391 Indexers behaviour made more consistent regarding length of indexed words...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4091
diff changeset
70 % (self.minlength, self.maxlength), text)]
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4008
diff changeset
71 words = set()
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
72 for word in wordlist:
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3414
diff changeset
73 if self.is_stopword(word): continue
3617
f12722c7b9ee improvements
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
74 words.add(word)
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
75
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
76 # for each word, add an entry in the db
3617
f12722c7b9ee improvements
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
77 sql = 'insert into __words (_word, _textid) values (%s, %s)'%(a, a)
f12722c7b9ee improvements
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
78 words = [(word, id) for word in words]
3618
b31a2e35be80 pysqlite 1.1.6 does not allow to pass a list of tuples to cursor.execute().
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3617
diff changeset
79 self.db.cursor.executemany(sql, words)
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
80
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
81 def find(self, wordlist):
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4008
diff changeset
82 """look up all the words in the wordlist.
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
83 If none are found return an empty dictionary
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
84 * more rules here
4089
eddb82d0964c Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents: 4008
diff changeset
85 """
3033
f8d0fd056ac0 fix indexer searching with no valid words [SF#1086787]
Richard Jones <richard@users.sourceforge.net>
parents: 2872
diff changeset
86 if not wordlist:
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3988
diff changeset
87 return []
3033
f8d0fd056ac0 fix indexer searching with no valid words [SF#1086787]
Richard Jones <richard@users.sourceforge.net>
parents: 2872
diff changeset
88
4252
2ff6f39aa391 Indexers behaviour made more consistent regarding length of indexed words...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4091
diff changeset
89 l = [word.upper() for word in wordlist
2ff6f39aa391 Indexers behaviour made more consistent regarding length of indexed words...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4091
diff changeset
90 if self.minlength <= len(word) <= self.maxlength]
2ff6f39aa391 Indexers behaviour made more consistent regarding length of indexed words...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents: 4091
diff changeset
91 l = [word for word in l if not self.is_stopword(word)]
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
92
3048
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
93 if not l:
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3988
diff changeset
94 return []
3048
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
95
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
96 if self.db.implements_intersect:
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
97 # simple AND search
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
98 sql = 'select distinct(_textid) from __words where _word=%s'%self.db.arg
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
99 sql = '\nINTERSECT\n'.join([sql]*len(l))
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
100 self.db.cursor.execute(sql, tuple(l))
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
101 r = self.db.cursor.fetchall()
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
102 if not r:
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3988
diff changeset
103 return []
3048
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
104 a = ','.join([self.db.arg] * len(r))
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
105 sql = 'select _class, _itemid, _prop from __textids '\
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
106 'where _textid in (%s)'%a
3718
0d561b24ceff support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents: 3618
diff changeset
107 self.db.cursor.execute(sql, tuple([int(row[0]) for row in r]))
3048
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
108
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
109 else:
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
110 # A more complex version for MySQL since it doesn't implement INTERSECT
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
111
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
112 # Construct SQL statement to join __words table to itself
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
113 # multiple times.
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
114 sql = """select distinct(__words1._textid)
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
115 from __words as __words1 %s
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
116 where __words1._word=%s %s"""
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
117
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
118 join_tmpl = ' left join __words as __words%d using (_textid) \n'
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
119 match_tmpl = ' and __words%d._word=%s \n'
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
120
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
121 join_list = []
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
122 match_list = []
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
123 for n in xrange(len(l) - 1):
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
124 join_list.append(join_tmpl % (n + 2))
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
125 match_list.append(match_tmpl % (n + 2, self.db.arg))
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
126
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
127 sql = sql%(' '.join(join_list), self.db.arg, ' '.join(match_list))
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
128 self.db.cursor.execute(sql, l)
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
129
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
130 r = map(lambda x: x[0], self.db.cursor.fetchall())
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
131 if not r:
4008
0bf9f8ae7d1b fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents: 3988
diff changeset
132 return []
3048
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
133
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
134 a = ','.join([self.db.arg] * len(r))
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
135 sql = 'select _class, _itemid, _prop from __textids '\
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
136 'where _textid in (%s)'%a
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
137
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
138 self.db.cursor.execute(sql, tuple(map(int, r)))
d9b4224f955c merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents: 3033
diff changeset
139
3076
2817a4db901d Change indexer_common.search() to take a list of nodeids...
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 3048
diff changeset
140 return self.db.cursor.fetchall()
2093
3f6024ab2c7a That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
141
Roundup Issue Tracker: http://roundup-tracker.org/