Mercurial > p > roundup > code

annotate roundup/roundup_indexer.py @ 692:0521ddc7bb31 search_indexing-0-4-2-branch

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Miscellaneous changes. cgi_client.py removed search link for the time being moved rendering of matches to htmltemplate hyperdb.py filtering of nodes on full text search incorporated in filter method roundupdb.py added paramater to call of filter method roundup_indexer.py added search method to RoundupIndexer class
author Roche Compaan <rochecompaan@users.sourceforge.net>
date Fri, 19 Apr 2002 19:54:42 +0000
parents 1b2d0e702ca8
children 8d2cb0d09da4
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
681
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
1 #
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
2 # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/)
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
3 # This module is free software, and you may redistribute it and/or modify
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
4 # under the same terms as Python, so long as this copyright message and
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
5 # disclaimer are retained in their original form.
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
6 #
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
7 # IN NO EVENT SHALL BIZAR SOFTWARE PTY LTD BE LIABLE TO ANY PARTY FOR
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
8 # DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
9 # OUT OF THE USE OF THIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
10 # POSSIBILITY OF SUCH DAMAGE.
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
11 #
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
12 # BIZAR SOFTWARE PTY LTD SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING,
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
13 # BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
14 # FOR A PARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN "AS IS"
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
15 # BASIS, AND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE,
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
16 # SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
17 #
692
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
18 #$Id: roundup_indexer.py,v 1.1.2.2 2002-04-19 19:54:42 rochecompaan Exp $
681
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
19 '''
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
20 This module provides an indexer class, RoundupIndexer, that stores text
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
21 indices in a roundup instance. This class makes searching the content of
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
22 messages and text files possible.
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
23 '''
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
24 import os
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
25 from roundup.indexer import SlicedZPickleIndexer
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
26
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
27 class RoundupIndexer(SlicedZPickleIndexer):
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
28 ''' Indexes messages and files
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
29 '''
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
30
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
31 def __init__(self, db_path):
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
32 indexdb_path = os.path.join(db_path, 'indexes')
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
33 index_exists = 0
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
34 if not os.path.exists(indexdb_path):
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
35 os.makedirs(indexdb_path)
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
36 os.chmod(indexdb_path, 0775)
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
37 index_exists = 1
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
38 index_path = os.path.join(indexdb_path, 'index.db')
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
39 SlicedZPickleIndexer.__init__(self,
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
40 INDEXDB=index_path, QUIET=9)
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
41 files_path = os.path.join(db_path, 'files')
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
42 if not index_exists:
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
43 self.add_files(dir=files_path)
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
44 self.save_index()
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
45
692
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
46 def search(self, search_terms, klass):
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
47 ''' display search results
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
48 '''
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
49 hits = self.find(search_terms)
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
50 links = []
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
51 nodeids = {}
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
52 designator_propname = {'msg': 'messages',
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
53 'file': 'files'}
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
54 if hits:
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
55 hitcount = len(hits)
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
56 # build a dictionary of nodes and their associated messages
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
57 # and files
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
58 for hit in hits.keys():
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
59 filename = hits[hit].split('/')[-1]
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
60 for designator, propname in designator_propname.items():
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
61 if filename.find(designator) == -1: continue
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
62 nodeid = filename[len(designator):]
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
63 result = apply(klass.find, (), {propname:nodeid})
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
64 if not result: continue
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
65
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
66 id = str(result[0])
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
67 if not nodeids.has_key(id):
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
68 nodeids[id] = {}
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
69
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
70 node_dict = nodeids[id]
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
71 if not node_dict.has_key(propname):
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
72 node_dict[propname] = [nodeid]
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
73 elif node_dict.has_key(propname):
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
74 node_dict[propname].append(nodeid)
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
75
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
76 return nodeids
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
77
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
78
681
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
79 #
1b2d0e702ca8 Added feature [SF#526730] - search for messages capability
Roche Compaan <rochecompaan@users.sourceforge.net>
parents:
diff changeset
80 #$Log: not supported by cvs2svn $
692
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
81 #Revision 1.1.2.1 2002/04/03 11:55:57 rochecompaan
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
82 # . Added feature #526730 - search for messages capability
0521ddc7bb31 Miscellaneous changes.
Roche Compaan <rochecompaan@users.sourceforge.net>
parents: 681
diff changeset
83 #
Roundup Issue Tracker: http://roundup-tracker.org/