comparison roundup/mailgw.py @ 110:19686b60e410

Multipart message class has the getPart method now. Added some tests for it.
author Richard Jones <richard@users.sourceforge.net>
date Sat, 28 Jul 2001 06:43:02 +0000
parents 381016730332
children 0791d13baea7
comparison
equal deleted inserted replaced
109:00b13b82adc1 110:19686b60e410
1 ''' 1 '''
2 Incoming messages are examined for multiple parts. In a multipart/mixed 2 An e-mail gateway for Roundup.
3 message or part, each subpart is extracted and examined. In a 3
4 multipart/alternative message or part, we look for a text/plain subpart and 4 Incoming messages are examined for multiple parts:
5 ignore the other parts. The text/plain subparts are assembled to form the 5 . In a multipart/mixed message or part, each subpart is extracted and
6 textual body of the message, to be stored in the file associated with a 6 examined. The text/plain subparts are assembled to form the textual
7 "msg" class node. Any parts of other types are each stored in separate 7 body of the message, to be stored in the file associated with a "msg"
8 files and given "file" class nodes that are linked to the "msg" node. 8 class node. Any parts of other types are each stored in separate files
9 9 and given "file" class nodes that are linked to the "msg" node.
10 . In a multipart/alternative message or part, we look for a text/plain
11 subpart and ignore the other parts.
12
13 Summary
14 -------
10 The "summary" property on message nodes is taken from the first non-quoting 15 The "summary" property on message nodes is taken from the first non-quoting
11 section in the message body. The message body is divided into sections by 16 section in the message body. The message body is divided into sections by
12 blank lines. Sections where the second and all subsequent lines begin with 17 blank lines. Sections where the second and all subsequent lines begin with
13 a ">" or "|" character are considered "quoting sections". The first line of 18 a ">" or "|" character are considered "quoting sections". The first line of
14 the first non-quoting section becomes the summary of the message. 19 the first non-quoting section becomes the summary of the message.
15 20
21 Addresses
22 ---------
16 All of the addresses in the To: and Cc: headers of the incoming message are 23 All of the addresses in the To: and Cc: headers of the incoming message are
17 looked up among the user nodes, and the corresponding users are placed in 24 looked up among the user nodes, and the corresponding users are placed in
18 the "recipients" property on the new "msg" node. The address in the From: 25 the "recipients" property on the new "msg" node. The address in the From:
19 header similarly determines the "author" property of the new "msg" 26 header similarly determines the "author" property of the new "msg"
20 node. The default handling for addresses that don't have corresponding 27 node. The default handling for addresses that don't have corresponding
22 address. (The web interface does not permit logins for users with no 29 address. (The web interface does not permit logins for users with no
23 passwords.) If we prefer to reject mail from outside sources, we can simply 30 passwords.) If we prefer to reject mail from outside sources, we can simply
24 register an auditor on the "user" class that prevents the creation of user 31 register an auditor on the "user" class that prevents the creation of user
25 nodes with no passwords. 32 nodes with no passwords.
26 33
34 Actions
35 -------
27 The subject line of the incoming message is examined to determine whether 36 The subject line of the incoming message is examined to determine whether
28 the message is an attempt to create a new item or to discuss an existing 37 the message is an attempt to create a new item or to discuss an existing
29 item. A designator enclosed in square brackets is sought as the first thing 38 item. A designator enclosed in square brackets is sought as the first thing
30 on the subject line (after skipping any "Fwd:" or "Re:" prefixes). 39 on the subject line (after skipping any "Fwd:" or "Re:" prefixes).
31 40
36 If just an item class name is found there, we attempt to create a new item 45 If just an item class name is found there, we attempt to create a new item
37 of that class with its "messages" property initialized to contain the new 46 of that class with its "messages" property initialized to contain the new
38 "msg" node and its "files" property initialized to contain any new "file" 47 "msg" node and its "files" property initialized to contain any new "file"
39 nodes. 48 nodes.
40 49
50 Triggers
51 --------
41 Both cases may trigger detectors (in the first case we are calling the 52 Both cases may trigger detectors (in the first case we are calling the
42 set() method to add the message to the item's spool; in the second case we 53 set() method to add the message to the item's spool; in the second case we
43 are calling the create() method to create a new node). If an auditor raises 54 are calling the create() method to create a new node). If an auditor raises
44 an exception, the original message is bounced back to the sender with the 55 an exception, the original message is bounced back to the sender with the
45 explanatory message given in the exception. 56 explanatory message given in the exception.
46 57
47 $Id: mailgw.py,v 1.3 2001-07-28 00:34:34 richard Exp $ 58 $Id: mailgw.py,v 1.4 2001-07-28 06:43:02 richard Exp $
48 ''' 59 '''
49 60
50 61
51 import string, re, os, mimetools, StringIO, smtplib, socket, binascii, quopri 62 import string, re, os, mimetools, cStringIO, smtplib, socket, binascii, quopri
52 import traceback 63 import traceback
53 import date 64 import date
54 65
55 def getPart(fp, boundary): 66 class Message(mimetools.Message):
56 line = '' 67 ''' subclass mimetools.Message so we can retrieve the parts of the
57 s = StringIO.StringIO() 68 message...
58 while 1: 69 '''
59 line_n = fp.readline() 70 def getPart(self):
60 if not line_n: 71 ''' Get a single part of a multipart message and return it as a new
61 break 72 Message instance.
62 line = line_n.strip() 73 '''
63 if line == '--'+boundary+'--': 74 boundary = self.getparam('boundary')
64 break 75 mid, end = '--'+boundary, '--'+boundary+'--'
65 if line == '--'+boundary: 76 s = cStringIO.StringIO()
66 break 77 while 1:
67 s.write(line_n) 78 line = self.fp.readline()
68 if not s.getvalue().strip(): 79 if not line:
69 return None 80 break
70 return s 81 if line.strip() in (mid, end):
82 break
83 s.write(line)
84 if not s.getvalue().strip():
85 return None
86 s.seek(0)
87 return Message(s)
71 88
72 subject_re = re.compile(r'(\[?(fwd|re):\s*)*' 89 subject_re = re.compile(r'(\[?(fwd|re):\s*)*'
73 r'(\[(?P<classname>[^\d]+)(?P<nodeid>\d+)?\])' 90 r'(\[(?P<classname>[^\d]+)(?P<nodeid>\d+)?\])'
74 r'(?P<title>[^\[]+)(\[(?P<args>.+?)\])?', re.I) 91 r'(?P<title>[^\[]+)(\[(?P<args>.+?)\])?', re.I)
75 92
76 class MailGW: 93 class MailGW:
77 def __init__(self, db): 94 def __init__(self, db):
78 self.db = db 95 self.db = db
79 96
80 def main(self, fp): 97 def main(self, fp):
98 ''' fp - the file from which to read the Message.
99
100 Read a message from fp and then call handle_message() with the
101 result. This method's job is to make that call and handle any
102 errors in a sane manner. It should be replaced if you wish to
103 handle errors in a different manner.
104 '''
81 # ok, figure the subject, author, recipients and content-type 105 # ok, figure the subject, author, recipients and content-type
82 message = mimetools.Message(fp) 106 message = Message(fp)
83 try: 107 try:
84 self.handle_message(message) 108 self.handle_message(message)
85 except: 109 except:
86 # bounce the message back to the sender with the error message 110 # bounce the message back to the sender with the error message
87 sendto = [message.getaddrlist('from')[0][1]] 111 sendto = [message.getaddrlist('from')[0][1]]
88 m = ['Subject: failed issue tracker submission'] 112 m = ['Subject: failed issue tracker submission']
89 m.append('') 113 m.append('')
90 # TODO as attachments? 114 # TODO as attachments?
91 m.append('---- traceback of failure ----') 115 m.append('---- traceback of failure ----')
92 s = StringIO.StringIO() 116 s = cStringIO.StringIO()
93 import traceback 117 import traceback
94 traceback.print_exc(None, s) 118 traceback.print_exc(None, s)
95 m.append(s.getvalue()) 119 m.append(s.getvalue())
96 m.append('---- failed message follows ----') 120 m.append('---- failed message follows ----')
97 try: 121 try:
106 return "Couldn't send confirmation email: mailhost %s"%value 130 return "Couldn't send confirmation email: mailhost %s"%value
107 except smtplib.SMTPException, value: 131 except smtplib.SMTPException, value:
108 return "Couldn't send confirmation email: %s"%value 132 return "Couldn't send confirmation email: %s"%value
109 133
110 def handle_message(self, message): 134 def handle_message(self, message):
135 ''' message - a Message instance
136
137 Parse the message as per the module docstring.
138 '''
111 # handle the subject line 139 # handle the subject line
112 m = subject_re.match(message.getheader('subject')) 140 m = subject_re.match(message.getheader('subject'))
113 if not m: 141 if not m:
114 raise ValueError, 'No [designator] found in subject "%s"' 142 raise ValueError, 'No [designator] found in subject "%s"'
115 classname = m.group('classname') 143 classname = m.group('classname')
148 176
149 # now handle the body - find the message 177 # now handle the body - find the message
150 content_type = message.gettype() 178 content_type = message.gettype()
151 attachments = [] 179 attachments = []
152 if content_type == 'multipart/mixed': 180 if content_type == 'multipart/mixed':
153 boundary = message.getparam('boundary')
154 # skip over the intro to the first boundary 181 # skip over the intro to the first boundary
155 part = getPart(message.fp, boundary) 182 part = message.getPart()
156 content = None 183 content = None
157 while 1: 184 while 1:
158 # get the next part 185 # get the next part
159 part = getPart(message.fp, boundary) 186 part = message.getPart()
160 if part is None: 187 if part is None:
161 break 188 break
162 # parse it 189 # parse it
163 part.seek(0) 190 subtype = part.gettype()
164 submessage = mimetools.Message(part)
165 subtype = submessage.gettype()
166 if subtype == 'text/plain' and not content: 191 if subtype == 'text/plain' and not content:
167 # this one's our content 192 # add all text/plain parts to the message content
168 content = part.read() 193 if content is None:
194 content = part.fp.read()
195 else:
196 content = content + part.fp.read()
197
169 elif subtype == 'message/rfc822': 198 elif subtype == 'message/rfc822':
170 i = part.tell() 199 # handle message/rfc822 specially - the name should be
171 subsubmess = mimetools.Message(part) 200 # the subject of the actual e-mail embedded here
172 name = subsubmess.getheader('subject') 201 i = part.fp.tell()
173 part.seek(i) 202 mailmess = Message(part.fp)
174 attachments.append((name, 'message/rfc822', part.read())) 203 name = mailmess.getheader('subject')
204 part.fp.seek(i)
205 attachments.append((name, 'message/rfc822', part.fp.read()))
206
175 else: 207 else:
176 # try name on Content-Type 208 # try name on Content-Type
177 name = submessage.getparam('name') 209 name = part.getparam('name')
178 # this is just an attachment 210 # this is just an attachment
179 data = part.read() 211 data = part.fp.read()
180 encoding = submessage.getencoding() 212 encoding = part.getencoding()
181 if encoding == 'base64': 213 if encoding == 'base64':
182 data = binascii.a2b_base64(data) 214 data = binascii.a2b_base64(data)
183 elif encoding == 'quoted-printable': 215 elif encoding == 'quoted-printable':
184 data = quopri.decode(data) 216 data = quopri.decode(data)
185 elif encoding == 'uuencoded': 217 elif encoding == 'uuencoded':
186 data = binascii.a2b_uu(data) 218 data = binascii.a2b_uu(data)
187 attachments.append((name, submessage.gettype(), data)) 219 attachments.append((name, part.gettype(), data))
220
188 if content is None: 221 if content is None:
189 raise ValueError, 'No text/plain part found' 222 raise ValueError, 'No text/plain part found'
190 223
191 elif content_type[:10] == 'multipart/': 224 elif content_type[:10] == 'multipart/':
192 boundary = message.getparam('boundary')
193 # skip over the intro to the first boundary 225 # skip over the intro to the first boundary
194 getPart(message.fp, boundary) 226 message.getPart()
195 content = None 227 content = None
196 while 1: 228 while 1:
197 # get the next part 229 # get the next part
198 part = getPart(message.fp, boundary) 230 part = message.getPart()
199 if part is None: 231 if part is None:
200 break 232 break
201 # parse it 233 # parse it
202 part.seek(0) 234 if part.gettype() == 'text/plain' and not content:
203 submessage = mimetools.Message(part)
204 if submessage.gettype() == 'text/plain' and not content:
205 # this one's our content 235 # this one's our content
206 content = part.read() 236 content = part.fp.read()
207 if content is None: 237 if content is None:
208 raise ValueError, 'No text/plain part found' 238 raise ValueError, 'No text/plain part found'
209 239
210 elif content_type != 'text/plain': 240 elif content_type != 'text/plain':
211 raise ValueError, 'No text/plain part found' 241 raise ValueError, 'No text/plain part found'
265 props['nosy'].sort() 295 props['nosy'].sort()
266 nodeid = cl.create(**props) 296 nodeid = cl.create(**props)
267 297
268 # 298 #
269 # $Log: not supported by cvs2svn $ 299 # $Log: not supported by cvs2svn $
300 # Revision 1.3 2001/07/28 00:34:34 richard
301 # Fixed some non-string node ids.
302 #
270 # Revision 1.2 2001/07/22 12:09:32 richard 303 # Revision 1.2 2001/07/22 12:09:32 richard
271 # Final commit of Grande Splite 304 # Final commit of Grande Splite
272 # 305 #
273 # 306 #

Roundup Issue Tracker: http://roundup-tracker.org/