Mercurial > p > roundup > code

annotate detectors/newitemcopy.py @ 6610:db3f0ba75b4a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Change checkpoint_data and restore_connection_on_error to subtransaction checkpoint_data and restore_connection_on_error used to commit() and rollback() the db connection. This causes additional I/O and load. Changed them to use 'SAVEPOINT name' and 'ROLLBACK TO name' to get a faster method for handling errors within a tranaction. One thing to note is that postgresql (unlike SQL std) doesn't overwrite an older savepoint with he same name. It keeps all savepoints but only rolls back to the newest one with a given name. This could be a resource issue. I left a commented out release statement in case somebody runs into an issue due to too many savepoints. I expect it to slow down the import but....
author John Rouillard <rouilj@ieee.org>
date Sat, 29 Jan 2022 11:29:36 -0500
parents 23b8e6067f7c
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
5115
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
1 from roundup import hyperdb, roundupdb
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
2 from roundup.mailer import Mailer
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
3
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
4
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
5 def indentChangeNoteValue(text):
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
6 # copied from roundupdb.IssueClass.indentChangeNoteValue()
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
7 lines = text.rstrip('\n').split('\n')
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
8 lines = [ ' '+line for line in lines ]
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
9 return '\n'.join(lines)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
10
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
11 def generateCreateNote(db, cl, nodeid):
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
12 # copied from roundupdb.IssueClass.generateCreateNote()
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
13 cn = cl.classname
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
14 props = cl.getprops(protected=0)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
15
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
16 # list the values
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
17 m = []
5395
23b8e6067f7c Python 3 preparation: update calls to dict methods.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5248
diff changeset
18 prop_items = sorted(props.items())
5115
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
19 for propname, prop in prop_items:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
20 value = cl.get(nodeid, propname, None)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
21 # skip boring entries
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
22 if not value:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
23 continue
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
24 if isinstance(prop, hyperdb.Link):
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
25 link = db.classes[prop.classname]
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
26 if value:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
27 key = link.labelprop(default_to_id=1)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
28 if key:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
29 value = link.get(value, key)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
30 else:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
31 value = ''
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
32 elif isinstance(prop, hyperdb.Multilink):
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
33 if value is None: value = []
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
34 l = []
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
35 link = db.classes[prop.classname]
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
36 key = link.labelprop(default_to_id=1)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
37 if key:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
38 value = [link.get(entry, key) for entry in value]
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
39 value.sort()
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
40 value = ', '.join(value)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
41 else:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
42 value = str(value)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
43 if '\n' in value:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
44 value = '\n'+indentChangeNoteValue(value)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
45 m.append('%s: %s'%(propname, value))
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
46 m.insert(0, '----------')
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
47 m.insert(0, '')
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
48 return '\n'.join(m)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
49
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
50 def newitemcopy(db, cl, nodeid, oldvalues):
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
51 ''' Copy a message about new items to the dispatcher address.
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
52 '''
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
53 try:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
54 create_note = cl.generateCreateNote(nodeid)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
55 except AttributeError:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
56 create_note = generateCreateNote(db, cl, nodeid)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
57
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
58 try:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
59 dispatcher_email = getattr(db.config, 'DISPATCHER_EMAIL')
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
60 except AttributeError:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
61 return
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
62
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
63 try:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
64 msgids = cl.get(nodeid, 'messages')
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
65 except KeyError:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
66 msgids = None
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
67
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
68 if msgids:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
69 # send a copy to the dispatcher
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
70 for msgid in msgids:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
71 try:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
72 cl.send_message(nodeid, msgid, create_note, [dispatcher_email])
5248
198b6e810c67 Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents: 5115
diff changeset
73 except roundupdb.MessageSendError as message:
198b6e810c67 Use Python-3-compatible 'as' syntax for except statements
Eric S. Raymond <esr@thyrsus.com>
parents: 5115
diff changeset
74 raise roundupdb.DetectorError(message)
5115
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
75 else:
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
76 mailer = Mailer(db.config)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
77 subject = 'New %s%s' % (cl.classname, nodeid)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
78 mailer.standard_message([dispatcher_email], subject, create_note)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
79
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
80 def init(db):
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
81 for classname in db.getclasses():
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
82 cl = db.getclass(classname)
ec06bd6ea156 issue2550767: Add newitemcopy.py detector to notify users of new
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
83 cl.react('create', newitemcopy)
Roundup Issue Tracker: http://roundup-tracker.org/