Mercurial > p > roundup > code
annotate detectors/irker.py @ 6593:e70e2789bc2c
issue2551189 - increase text search maxlength
This removes I think all the magic references to 25 and 30 (varchar
size) and replaces them with references to maxlength or maxlength+5.
I am not sure why the db column is 5 characters larger than the size
of what should be the max size of a word, but I'll keep the buffer
of 5 as making it 1/5 the size of maxlength makes less sense.
Also added tests for fts search in templating which were missing.
Added postgres, mysql and sqlite native indexing backends in which to
test fts. Added fts test to native-fts as well to make sure it's
working.
I want to commit this now for CI.
Todo:
add test cases for the use of FTS in the csv output in
actions.py. There is no test coverage of the match case there.
change maxlength to a higher value (50) as requested in the ticket.
Modify existing extremewords test cases to allow words > 25 and < 51
write code to migrate column sizes for mysql and postgresql to match
maxlength I will roll this into the version 7 schema update that
supports use of database fts support.
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Tue, 25 Jan 2022 13:22:00 -0500 |
| parents | 64b05e24dbd8 |
| children | 72f549e68534 |
| rev | line source |
|---|---|
|
4695
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
1 # This detector sends notification on IRC through an irker daemon |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
2 # (http://www.catb.org/esr/irker/) when issues are created or messages |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
3 # are added. |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
4 # |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
5 # Written by Ezio Melotti |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
6 # |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
7 # Requires a running irkerd daemon to work. See the irker documentation |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
8 # for more information about installing, configuring, and running irker. |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
9 # |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
10 # Add the IRC channel(s) that should receive notifications in |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
11 # detectors/config.ini as a comma-separated list, using this format: |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
12 # |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
13 # [irker] |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
14 # channels = irc://chat.freenode.net/channelname |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
15 # |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
16 |
|
5376
64b05e24dbd8
Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents:
4695
diff
changeset
|
17 from __future__ import print_function |
|
4695
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
18 import re |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
19 import json |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
20 import socket |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
21 |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
22 IRKER_HOST = 'localhost' |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
23 IRKER_PORT = 6659 |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
24 |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
25 max_content = 120 |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
26 |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
27 TEMPLATE = ('%(green)s%(author)s%(reset)s ' |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
28 '%(bluish)s#%(nodeid)s%(reset)s/%(title)s%(bold)s:%(bold)s ' |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
29 '%(log)s %(url)s') |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
30 |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
31 |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
32 def sendmsg(msg): |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
33 sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
34 try: |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
35 sock.connect((IRKER_HOST, IRKER_PORT)) |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
36 sock.sendall(msg + "\n") |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
37 finally: |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
38 sock.close() |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
39 |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
40 |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
41 def notify_irker(db, cl, nodeid, oldvalues): |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
42 messages = set(cl.get(nodeid, 'messages')) |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
43 if oldvalues: |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
44 messages -= set(oldvalues.get('messages', ())) |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
45 if not messages: |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
46 return |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
47 messages = list(messages) |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
48 |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
49 if oldvalues: |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
50 oldstatus = oldvalues['status'] |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
51 else: |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
52 oldstatus = None |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
53 newstatus = db.issue.get(nodeid, 'status') |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
54 if oldstatus != newstatus: |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
55 if oldvalues: |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
56 status = db.status.get(newstatus, 'name') |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
57 else: |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
58 status = 'new' |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
59 log = '[' + status + '] ' |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
60 else: |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
61 log = '' |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
62 for msg in messages: |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
63 log += db.msg.get(msg, 'content') |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
64 if len(log) > max_content: |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
65 log = log[:max_content-3] + '...' |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
66 log = re.sub('\s+', ' ', log) |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
67 |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
68 # include irc colors |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
69 params = { |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
70 'bold': '\x02', |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
71 'green': '\x0303', |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
72 'blue': '\x0302', |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
73 'bluish': '\x0310', |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
74 'yellow': '\x0307', |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
75 'brown': '\x0305', |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
76 'reset': '\x0F' |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
77 } |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
78 # extend with values used in the template |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
79 params['author'] = db.user.get(db.getuid(), 'username') |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
80 params['nodeid'] = nodeid |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
81 params['title'] = db.issue.get(nodeid, 'title') |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
82 params['log'] = log |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
83 params['url'] = '%sissue%s' % (db.config.TRACKER_WEB, nodeid) |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
84 |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
85 # create the message and use the list of channels defined in |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
86 # detectors/config.ini |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
87 msg = json.dumps({ |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
88 'to': db.config.detectors.IRKER_CHANNELS.split(','), |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
89 'privmsg': TEMPLATE % params, |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
90 }) |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
91 |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
92 try: |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
93 sendmsg(msg) |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
94 except Exception as e: |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
95 # Ignore any errors in sending the irker; |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
96 # if the server is down, that's just bad luck |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
97 # XXX might want to do some logging here |
|
5376
64b05e24dbd8
Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents:
4695
diff
changeset
|
98 print('* Sending message to irker failed', str(e)) |
|
4695
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
99 |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
100 def init(db): |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
101 db.issue.react('create', notify_irker) |
|
132650f4700a
#2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff
changeset
|
102 db.issue.react('set', notify_irker) |
