Mercurial > p > roundup > code

annotate detectors/irker.py @ 5305:e20f472fde7d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
issue2550799: provide basic support for handling html only emails Initial implementation and testing with the dehtml html converter done. The use of beautifulsoup 4 is not tested. My test system breaks when running dehtml.py using beautiful soup. I don't get the failures when running under the test harness, but the text output is significantly different (different line breaks, number of newlines etc.) The tests for dehtml need to be generated for beautiful soup and the expected output changed. Since I have a wonky install of beautiful soup, I don't trust my output as the standard to test against. Also since beautiful soup is optional, the test harness needs to skip the beautifulsoup tests if import bs4 fails. Again something outside of my expertise. I deleted the work I had done to implement that. I could not get it working and wanted to get this feature in in some form.
author John Rouillard <rouilj@ieee.org>
date Fri, 13 Oct 2017 21:46:59 -0400
parents 132650f4700a
children 64b05e24dbd8
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4695
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
1 # This detector sends notification on IRC through an irker daemon
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
2 # (http://www.catb.org/esr/irker/) when issues are created or messages
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
3 # are added.
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
4 #
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
5 # Written by Ezio Melotti
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
6 #
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
7 # Requires a running irkerd daemon to work. See the irker documentation
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
8 # for more information about installing, configuring, and running irker.
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
9 #
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
10 # Add the IRC channel(s) that should receive notifications in
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
11 # detectors/config.ini as a comma-separated list, using this format:
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
12 #
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
13 # [irker]
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
14 # channels = irc://chat.freenode.net/channelname
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
15 #
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
16
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
17 import re
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
18 import json
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
19 import socket
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
20
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
21 IRKER_HOST = 'localhost'
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
22 IRKER_PORT = 6659
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
23
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
24 max_content = 120
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
25
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
26 TEMPLATE = ('%(green)s%(author)s%(reset)s '
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
27 '%(bluish)s#%(nodeid)s%(reset)s/%(title)s%(bold)s:%(bold)s '
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
28 '%(log)s %(url)s')
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
29
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
30
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
31 def sendmsg(msg):
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
32 sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
33 try:
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
34 sock.connect((IRKER_HOST, IRKER_PORT))
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
35 sock.sendall(msg + "\n")
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
36 finally:
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
37 sock.close()
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
38
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
39
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
40 def notify_irker(db, cl, nodeid, oldvalues):
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
41 messages = set(cl.get(nodeid, 'messages'))
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
42 if oldvalues:
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
43 messages -= set(oldvalues.get('messages', ()))
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
44 if not messages:
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
45 return
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
46 messages = list(messages)
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
47
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
48 if oldvalues:
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
49 oldstatus = oldvalues['status']
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
50 else:
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
51 oldstatus = None
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
52 newstatus = db.issue.get(nodeid, 'status')
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
53 if oldstatus != newstatus:
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
54 if oldvalues:
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
55 status = db.status.get(newstatus, 'name')
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
56 else:
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
57 status = 'new'
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
58 log = '[' + status + '] '
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
59 else:
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
60 log = ''
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
61 for msg in messages:
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
62 log += db.msg.get(msg, 'content')
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
63 if len(log) > max_content:
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
64 log = log[:max_content-3] + '...'
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
65 log = re.sub('\s+', ' ', log)
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
66
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
67 # include irc colors
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
68 params = {
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
69 'bold': '\x02',
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
70 'green': '\x0303',
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
71 'blue': '\x0302',
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
72 'bluish': '\x0310',
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
73 'yellow': '\x0307',
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
74 'brown': '\x0305',
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
75 'reset': '\x0F'
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
76 }
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
77 # extend with values used in the template
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
78 params['author'] = db.user.get(db.getuid(), 'username')
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
79 params['nodeid'] = nodeid
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
80 params['title'] = db.issue.get(nodeid, 'title')
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
81 params['log'] = log
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
82 params['url'] = '%sissue%s' % (db.config.TRACKER_WEB, nodeid)
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
83
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
84 # create the message and use the list of channels defined in
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
85 # detectors/config.ini
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
86 msg = json.dumps({
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
87 'to': db.config.detectors.IRKER_CHANNELS.split(','),
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
88 'privmsg': TEMPLATE % params,
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
89 })
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
90
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
91 try:
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
92 sendmsg(msg)
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
93 except Exception as e:
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
94 # Ignore any errors in sending the irker;
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
95 # if the server is down, that's just bad luck
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
96 # XXX might want to do some logging here
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
97 print '* Sending message to irker failed', str(e)
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
98
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
99 def init(db):
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
100 db.issue.react('create', notify_irker)
132650f4700a #2550782: Added a new irker detector to send notifications on IRC when an issue is created or messages are added.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
diff changeset
101 db.issue.react('set', notify_irker)
Roundup Issue Tracker: http://roundup-tracker.org/