Mercurial > p > roundup > code

annotate roundup/support.py @ 5305:e20f472fde7d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
issue2550799: provide basic support for handling html only emails Initial implementation and testing with the dehtml html converter done. The use of beautifulsoup 4 is not tested. My test system breaks when running dehtml.py using beautiful soup. I don't get the failures when running under the test harness, but the text output is significantly different (different line breaks, number of newlines etc.) The tests for dehtml need to be generated for beautiful soup and the expected output changed. Since I have a wonky install of beautiful soup, I don't trust my output as the standard to test against. Also since beautiful soup is optional, the test harness needs to skip the beautifulsoup tests if import bs4 fails. Again something outside of my expertise. I deleted the work I had done to implement that. I could not get it working and wanted to get this feature in in some form.
author John Rouillard <rouilj@ieee.org>
date Fri, 13 Oct 2017 21:46:59 -0400
parents ac0c117cd24c
children 64b05e24dbd8
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1 """Implements various support classes and functions used in a number of
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2 places in Roundup code.
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3 """
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
4
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
5 __docformat__ = 'restructuredtext'
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
6
3644
f35ece8f8ff7 added StringHTMLProperty wrapped() method to wrap long lines in issue display
Richard Jones <richard@users.sourceforge.net>
parents: 3636
diff changeset
7 import os, time, sys, re
3019
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
8
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
9 class TruthDict:
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
10 '''Returns True for valid keys, False for others.
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
11 '''
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
12 def __init__(self, keys):
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
13 if keys:
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
14 self.keys = {}
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
15 for col in keys:
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
16 self.keys[col] = 1
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
17 else:
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
18 self.__getitem__ = lambda name: 1
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
19
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
20 def __getitem__(self, name):
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
21 return self.keys.has_key(name)
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
22
3019
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
23 def ensureParentsExist(dest):
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
24 if not os.path.exists(os.path.dirname(dest)):
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
25 os.makedirs(os.path.dirname(dest))
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
26
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
27 class PrioList:
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
28 '''Manages a sorted list.
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
29
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
30 Currently only implements method 'append' and iteration from a
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
31 full list interface.
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
32 Implementation: We manage a "sorted" status and sort on demand.
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
33 Appending to the list will require re-sorting before use.
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
34 >>> p = PrioList()
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
35 >>> for i in 5,7,1,-1:
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
36 ... p.append(i)
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
37 ...
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
38 >>> for k in p:
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
39 ... print k
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
40 ...
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
41 -1
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
42 1
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
43 5
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
44 7
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
45
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
46 '''
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
47 def __init__(self):
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
48 self.list = []
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
49 self.sorted = True
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
50
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
51 def append(self, item):
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
52 self.list.append(item)
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
53 self.sorted = False
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
54
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
55 def __iter__(self):
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
56 if not self.sorted:
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
57 self.list.sort()
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
58 self.sorted = True
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
59 return iter(self.list)
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
60
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
61 class Progress:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
62 '''Progress display for console applications.
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
63
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
64 See __main__ block at end of file for sample usage.
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
65 '''
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
66 def __init__(self, info, sequence):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
67 self.info = info
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
68 self.sequence = iter(sequence)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
69 self.total = len(sequence)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
70 self.start = self.now = time.time()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
71 self.num = 0
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
72 self.stepsize = self.total / 100 or 1
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
73 self.steptimes = []
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
74 self.display()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
75
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
76 def __iter__(self): return self
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
77
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
78 def next(self):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
79 self.num += 1
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
80
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
81 if self.num > self.total:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
82 print self.info, 'done', ' '*(75-len(self.info)-6)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
83 sys.stdout.flush()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
84 return self.sequence.next()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
85
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
86 if self.num % self.stepsize:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
87 return self.sequence.next()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
88
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
89 self.display()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
90 return self.sequence.next()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
91
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
92 def display(self):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
93 # figure how long we've spent - guess how long to go
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
94 now = time.time()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
95 steptime = now - self.now
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
96 self.steptimes.insert(0, steptime)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
97 if len(self.steptimes) > 5:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
98 self.steptimes.pop()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
99 steptime = sum(self.steptimes) / len(self.steptimes)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
100 self.now = now
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
101 eta = steptime * ((self.total - self.num)/self.stepsize)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
102
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
103 # tell it like it is (or might be)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
104 if now - self.start > 3:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
105 M = eta / 60
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
106 H = M / 60
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
107 M = M % 60
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
108 S = eta % 60
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
109 if self.total:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
110 s = '%s %2d%% (ETA %02d:%02d:%02d)'%(self.info,
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
111 self.num * 100. / self.total, H, M, S)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
112 else:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
113 s = '%s 0%% (ETA %02d:%02d:%02d)'%(self.info, H, M, S)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
114 elif self.total:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
115 s = '%s %2d%%'%(self.info, self.num * 100. / self.total)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
116 else:
3589
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
117 s = '%s %d done'%(self.info, self.num)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
118 sys.stdout.write(s + ' '*(75-len(s)) + '\r')
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
119 sys.stdout.flush()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
120
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
121 # vim: set et sts=4 sw=4 :
Roundup Issue Tracker: http://roundup-tracker.org/