Mercurial > p > roundup > code
annotate roundup/backends/indexer_rdbms.py @ 5548:fea11d05110e
Avoid errors from selecting "no selection" on multilink (issue2550722).
As discussed in issue 2550722 there are various cases where selecting
"no selection" on a multilink can result in inappropriate errors from
Roundup:
* If selecting "no selection" produces a null edit (a value was set in
the multilink in an edit with an error, then removed again, along
with all other changes, in the next form submission), so the page is
rendered from the form contents including the "-<id>" value for "no
selection" for the multilink.
* If creating an item with a nonempty value for a multilink has an
error, and the resubmission changes that multilink to "no selection"
(and this in turn has subcases, according to whether the creation
then succeeds or fails on the resubmission, which need fixes in
different places in the Roundup code).
All of these cases have in common that it is expected and OK to have a
"-<id>" value for a submission for a multilink when <id> is not set in
that multilink in the database (because the original attempt to set
<id> in that multilink had an error), so the hyperdb.py logic to give
an error in that case is thus removed. In the subcase of the second
case where the resubmission with "no selection" has an error, the
templating code tries to produce a menu entry for the "-<id>"
multilink value, which also results in an error, hence the
templating.py change to ignore such values in the list for a
multilink.
| author | Joseph Myers <jsm@polyomino.org.uk> |
|---|---|
| date | Thu, 27 Sep 2018 11:33:01 +0000 |
| parents | 56c9bcdea47f |
| children | 8bda74ee7070 |
| rev | line source |
|---|---|
|
4089
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
4008
diff
changeset
|
1 """ This implements the full-text indexer over two RDBMS tables. The first |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
2 is a mapping of words to occurance IDs. The second maps the IDs to (Class, |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
3 propname, itemid) instances. |
|
4089
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
4008
diff
changeset
|
4 """ |
|
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
4008
diff
changeset
|
5 import re |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
6 |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3414
diff
changeset
|
7 from roundup.backends.indexer_common import Indexer as IndexerBase |
|
5416
56c9bcdea47f
Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5391
diff
changeset
|
8 from roundup.anypy.strings import us2u, u2s |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
9 |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3414
diff
changeset
|
10 class Indexer(IndexerBase): |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
11 def __init__(self, db): |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3414
diff
changeset
|
12 IndexerBase.__init__(self, db) |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
13 self.db = db |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
14 self.reindex = 0 |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
15 |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
16 def close(self): |
|
4089
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
4008
diff
changeset
|
17 """close the indexing database""" |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
18 # just nuke the circular reference |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
19 self.db = None |
|
3295
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3092
diff
changeset
|
20 |
|
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3092
diff
changeset
|
21 def save_index(self): |
|
4089
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
4008
diff
changeset
|
22 """Save the changes to the index.""" |
|
3295
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3092
diff
changeset
|
23 # not necessary - the RDBMS connection will handle this for us |
|
a615cc230160
added Xapian indexer; replaces standard indexers if Xapian is available
Richard Jones <richard@users.sourceforge.net>
parents:
3092
diff
changeset
|
24 pass |
|
3331
7bc09d5d9544
perform word splitting in unicode for national characters support
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3295
diff
changeset
|
25 |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
26 def force_reindex(self): |
|
4089
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
4008
diff
changeset
|
27 """Force a reindexing of the database. This essentially |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
28 empties the tables ids and index and sets a flag so |
|
4089
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
4008
diff
changeset
|
29 that the databases are reindexed""" |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
30 self.reindex = 1 |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
31 |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
32 def should_reindex(self): |
|
4089
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
4008
diff
changeset
|
33 """returns True if the indexes need to be rebuilt""" |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
34 return self.reindex |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
35 |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
36 def add_text(self, identifier, text, mime_type='text/plain'): |
|
4089
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
4008
diff
changeset
|
37 """ "identifier" is (classname, itemid, property) """ |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
38 if mime_type != 'text/plain': |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
39 return |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
40 |
|
3988
b50446ff16f4
fix comment, thanks alex
Richard Jones <richard@users.sourceforge.net>
parents:
3981
diff
changeset
|
41 # Ensure all elements of the identifier are strings 'cos the itemid |
|
3981
82e116d515d2
another psql 8.3 compat fix
Richard Jones <richard@users.sourceforge.net>
parents:
3718
diff
changeset
|
42 # column is varchar even if item ids may be numbers elsewhere in the |
|
82e116d515d2
another psql 8.3 compat fix
Richard Jones <richard@users.sourceforge.net>
parents:
3718
diff
changeset
|
43 # code. ugh. |
|
4008
0bf9f8ae7d1b
fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents:
3988
diff
changeset
|
44 identifier = tuple(map(str, identifier)) |
|
3981
82e116d515d2
another psql 8.3 compat fix
Richard Jones <richard@users.sourceforge.net>
parents:
3718
diff
changeset
|
45 |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
46 # first, find the id of the (classname, itemid, property) |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
47 a = self.db.arg |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
48 sql = 'select _textid from __textids where _class=%s and '\ |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
49 '_itemid=%s and _prop=%s'%(a, a, a) |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
50 self.db.cursor.execute(sql, identifier) |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
51 r = self.db.cursor.fetchone() |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
52 if not r: |
|
3617
f12722c7b9ee
improvements
Richard Jones <richard@users.sourceforge.net>
parents:
3544
diff
changeset
|
53 # not previously indexed |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
54 id = self.db.newid('__textids') |
|
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
55 sql = 'insert into __textids (_textid, _class, _itemid, _prop)'\ |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
56 ' values (%s, %s, %s, %s)'%(a, a, a, a) |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
57 self.db.cursor.execute(sql, (id, ) + identifier) |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
58 else: |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
59 id = int(r[0]) |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
60 # clear out any existing indexed values |
|
2098
18addf2a8596
Implemented proper datatypes in mysql and postgresql backends...
Richard Jones <richard@users.sourceforge.net>
parents:
2093
diff
changeset
|
61 sql = 'delete from __words where _textid=%s'%a |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
62 self.db.cursor.execute(sql, (id, )) |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
63 |
|
3617
f12722c7b9ee
improvements
Richard Jones <richard@users.sourceforge.net>
parents:
3544
diff
changeset
|
64 # ok, find all the unique words in the text |
|
5416
56c9bcdea47f
Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5391
diff
changeset
|
65 text = us2u(text, "replace") |
|
4284
86e8b2a615fa
Fix Issue2550609, hopefully for real, this time.
Stefan Seefeld <stefan@seefeld.name>
parents:
4283
diff
changeset
|
66 text = text.upper() |
|
5416
56c9bcdea47f
Python 3 preparation: unicode.
Joseph Myers <jsm@polyomino.org.uk>
parents:
5391
diff
changeset
|
67 wordlist = [u2s(w) |
| 4282 | 68 for w in re.findall(r'(?u)\b\w{%d,%d}\b' |
| 69 % (self.minlength, self.maxlength), text)] | |
|
4089
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
4008
diff
changeset
|
70 words = set() |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
71 for word in wordlist: |
|
3544
5cd1c83dea50
Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents:
3414
diff
changeset
|
72 if self.is_stopword(word): continue |
|
3617
f12722c7b9ee
improvements
Richard Jones <richard@users.sourceforge.net>
parents:
3544
diff
changeset
|
73 words.add(word) |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
74 |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
75 # for each word, add an entry in the db |
|
3617
f12722c7b9ee
improvements
Richard Jones <richard@users.sourceforge.net>
parents:
3544
diff
changeset
|
76 sql = 'insert into __words (_word, _textid) values (%s, %s)'%(a, a) |
|
f12722c7b9ee
improvements
Richard Jones <richard@users.sourceforge.net>
parents:
3544
diff
changeset
|
77 words = [(word, id) for word in words] |
|
3618
b31a2e35be80
pysqlite 1.1.6 does not allow to pass a list of tuples to cursor.execute().
Alexander Smishlajev <a1s@users.sourceforge.net>
parents:
3617
diff
changeset
|
78 self.db.cursor.executemany(sql, words) |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
79 |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
80 def find(self, wordlist): |
|
4089
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
4008
diff
changeset
|
81 """look up all the words in the wordlist. |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
82 If none are found return an empty dictionary |
|
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
83 * more rules here |
|
4089
eddb82d0964c
Add compatibility package to allow us to deal with Python versions 2.3..2.6.
Richard Jones <richard@users.sourceforge.net>
parents:
4008
diff
changeset
|
84 """ |
|
3033
f8d0fd056ac0
fix indexer searching with no valid words [SF#1086787]
Richard Jones <richard@users.sourceforge.net>
parents:
2872
diff
changeset
|
85 if not wordlist: |
|
4008
0bf9f8ae7d1b
fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents:
3988
diff
changeset
|
86 return [] |
|
3033
f8d0fd056ac0
fix indexer searching with no valid words [SF#1086787]
Richard Jones <richard@users.sourceforge.net>
parents:
2872
diff
changeset
|
87 |
|
4252
2ff6f39aa391
Indexers behaviour made more consistent regarding length of indexed words...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4091
diff
changeset
|
88 l = [word.upper() for word in wordlist |
|
2ff6f39aa391
Indexers behaviour made more consistent regarding length of indexed words...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4091
diff
changeset
|
89 if self.minlength <= len(word) <= self.maxlength] |
|
2ff6f39aa391
Indexers behaviour made more consistent regarding length of indexed words...
Bernhard Reiter <Bernhard.Reiter@intevation.de>
parents:
4091
diff
changeset
|
90 l = [word for word in l if not self.is_stopword(word)] |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
91 |
|
3048
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
92 if not l: |
|
4008
0bf9f8ae7d1b
fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents:
3988
diff
changeset
|
93 return [] |
|
3048
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
94 |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
95 if self.db.implements_intersect: |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
96 # simple AND search |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
97 sql = 'select distinct(_textid) from __words where _word=%s'%self.db.arg |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
98 sql = '\nINTERSECT\n'.join([sql]*len(l)) |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
99 self.db.cursor.execute(sql, tuple(l)) |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
100 r = self.db.cursor.fetchall() |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
101 if not r: |
|
4008
0bf9f8ae7d1b
fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents:
3988
diff
changeset
|
102 return [] |
|
3048
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
103 a = ','.join([self.db.arg] * len(r)) |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
104 sql = 'select _class, _itemid, _prop from __textids '\ |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
105 'where _textid in (%s)'%a |
|
3718
0d561b24ceff
support sqlite3
Richard Jones <richard@users.sourceforge.net>
parents:
3618
diff
changeset
|
106 self.db.cursor.execute(sql, tuple([int(row[0]) for row in r])) |
|
3048
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
107 |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
108 else: |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
109 # A more complex version for MySQL since it doesn't implement INTERSECT |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
110 |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
111 # Construct SQL statement to join __words table to itself |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
112 # multiple times. |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
113 sql = """select distinct(__words1._textid) |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
114 from __words as __words1 %s |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
115 where __words1._word=%s %s""" |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
116 |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
117 join_tmpl = ' left join __words as __words%d using (_textid) \n' |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
118 match_tmpl = ' and __words%d._word=%s \n' |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
119 |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
120 join_list = [] |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
121 match_list = [] |
|
5391
a391a071d045
Python 3 preparation: use range() instead of xrange().
Joseph Myers <jsm@polyomino.org.uk>
parents:
4687
diff
changeset
|
122 for n in range(len(l) - 1): |
|
3048
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
123 join_list.append(join_tmpl % (n + 2)) |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
124 match_list.append(match_tmpl % (n + 2, self.db.arg)) |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
125 |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
126 sql = sql%(' '.join(join_list), self.db.arg, ' '.join(match_list)) |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
127 self.db.cursor.execute(sql, l) |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
128 |
|
4357
13b3155869e0
Beginnings of a big code cleanup / modernisation to make 2to3 happy
Richard Jones <richard@users.sourceforge.net>
parents:
4284
diff
changeset
|
129 r = [x[0] for x in self.db.cursor.fetchall()] |
|
3048
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
130 if not r: |
|
4008
0bf9f8ae7d1b
fix bug introduced in 1.4.5 in RDBMS full-text indexing;
Richard Jones <richard@users.sourceforge.net>
parents:
3988
diff
changeset
|
131 return [] |
|
3048
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
132 |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
133 a = ','.join([self.db.arg] * len(r)) |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
134 sql = 'select _class, _itemid, _prop from __textids '\ |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
135 'where _textid in (%s)'%a |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
136 |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
137 self.db.cursor.execute(sql, tuple(map(int, r))) |
|
d9b4224f955c
merge from maint-0-8
Richard Jones <richard@users.sourceforge.net>
parents:
3033
diff
changeset
|
138 |
|
3076
2817a4db901d
Change indexer_common.search() to take a list of nodeids...
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
3048
diff
changeset
|
139 return self.db.cursor.fetchall() |
|
2093
3f6024ab2c7a
That's the last of the RDBMS migration steps done! Yay!
Richard Jones <richard@users.sourceforge.net>
parents:
diff
changeset
|
140 |
