Mercurial > p > roundup > code

annotate detectors/newitemcopy.py @ 5543:bc3e00a3d24b

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
MySQL backend fixes for Python 3. With Python 2, text sent to and from MySQL is treated as bytes in Python. The database may be recorded by MySQL as having some other encoding (latin1 being the default in some MySQL versions - Roundup does not set an encoding explicitly, unlike in back_postgresql), but as long as MySQL's notion of the connection encoding agrees with its notion of the database encoding, no conversions actually take place and the bytes are stored and returned as-is. With Python 3, text sent to and from MySQL is treated as Python Unicode strings. When the database and connection encoding is latin1, that means the bytes stored in the database under Python 2 are interpreted as latin1 and converted from that to Unicode, producing incorrect results for any non-ASCII characters; furthermore, if trying to store new non-ASCII data in the database under Python 3, any non-latin1 characters produce errors. This patch arranges for both the connection and database character sets to be UTF-8 when using Python 3, and documents a need to export and import the database when moving from Python 2 to Python 3 with this backend.
author Joseph Myers <jsm@polyomino.org.uk>
date Sun, 16 Sep 2018 16:19:20 +0000
parents 23b8e6067f7c
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
5115
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
1 from roundup import hyperdb, roundupdb
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
2 from roundup.mailer import Mailer
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
3
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
4
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
5 def indentChangeNoteValue(text):
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
6 # copied from roundupdb.IssueClass.indentChangeNoteValue()
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
7 lines = text.rstrip('\n').split('\n')
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
8 lines = [ ' '+line for line in lines ]
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
9 return '\n'.join(lines)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
10
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
11 def generateCreateNote(db, cl, nodeid):
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
12 # copied from roundupdb.IssueClass.generateCreateNote()
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
13 cn = cl.classname
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
14 props = cl.getprops(protected=0)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
15
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
16 # list the values
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
17 m = []
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5248
diff changeset
18 prop_items = sorted(props.items())
5115
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
19 for propname, prop in prop_items:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
20 value = cl.get(nodeid, propname, None)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
21 # skip boring entries
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
22 if not value:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
23 continue
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
24 if isinstance(prop, hyperdb.Link):
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
25 link = db.classes[prop.classname]
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
26 if value:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
27 key = link.labelprop(default_to_id=1)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
28 if key:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
29 value = link.get(value, key)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
30 else:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
31 value = ''
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
32 elif isinstance(prop, hyperdb.Multilink):
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
33 if value is None: value = []
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
34 l = []
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
35 link = db.classes[prop.classname]
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
36 key = link.labelprop(default_to_id=1)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
37 if key:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
38 value = [link.get(entry, key) for entry in value]
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
39 value.sort()
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
40 value = ', '.join(value)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
41 else:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
42 value = str(value)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
43 if '\n' in value:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
44 value = '\n'+indentChangeNoteValue(value)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
45 m.append('%s: %s'%(propname, value))
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
46 m.insert(0, '----------')
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
47 m.insert(0, '')
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
48 return '\n'.join(m)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
49
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
50 def newitemcopy(db, cl, nodeid, oldvalues):
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
51 ''' Copy a message about new items to the dispatcher address.
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
52 '''
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
53 try:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
54 create_note = cl.generateCreateNote(nodeid)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
55 except AttributeError:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
56 create_note = generateCreateNote(db, cl, nodeid)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
57
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
58 try:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
59 dispatcher_email = getattr(db.config, 'DISPATCHER_EMAIL')
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
60 except AttributeError:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
61 return
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
62
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
63 try:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
64 msgids = cl.get(nodeid, 'messages')
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
65 except KeyError:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
66 msgids = None
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
67
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
68 if msgids:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
69 # send a copy to the dispatcher
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
70 for msgid in msgids:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
71 try:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
72 cl.send_message(nodeid, msgid, create_note, [dispatcher_email])
5248
198b6e810c67 Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents: 5115
diff changeset
73 except roundupdb.MessageSendError as message:
198b6e810c67 Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents: 5115
diff changeset
74 raise roundupdb.DetectorError(message)
5115
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
75 else:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
76 mailer = Mailer(db.config)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
77 subject = 'New %s%s' % (cl.classname, nodeid)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
78 mailer.standard_message([dispatcher_email], subject, create_note)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
79
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
80 def init(db):
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
81 for classname in db.getclasses():
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
82 cl = db.getclass(classname)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
83 cl.react('create', newitemcopy)
Roundup Issue Tracker: http://roundup-tracker.org/