Mercurial > p > roundup > code

annotate roundup/support.py @ 5400:2120f77554d5

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Python 3 preparation: use // and __truediv__ as needed. Tool-assisted patch. Those divisions that I thought must be integer floor divisions and rely on Python 2 integer floor division semantics are changed to use // (if any are actually meant to be floating-point divisions, that would break things). One __div__ method is changed to __truediv__ (with __div__ = __truediv__ for Python 2 compatibility).
author Joseph Myers <jsm@polyomino.org.uk>
date Tue, 24 Jul 2018 23:16:09 +0000
parents 831787cf6694
children afd9fd3a0edb
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1 """Implements various support classes and functions used in a number of
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2 places in Roundup code.
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3 """
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
4
5376
64b05e24dbd8 Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5043
diff changeset
5 from __future__ import print_function
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
6 __docformat__ = 'restructuredtext'
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
7
3644
f35ece8f8ff7 added StringHTMLProperty wrapped() method to wrap long lines in issue display
Richard Jones <richard@users.sourceforge.net>
parents: 3636
diff changeset
8 import os, time, sys, re
3019
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
9
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
10 class TruthDict:
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
11 '''Returns True for valid keys, False for others.
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
12 '''
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
13 def __init__(self, keys):
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
14 if keys:
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
15 self.keys = {}
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
16 for col in keys:
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
17 self.keys[col] = 1
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
18 else:
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
19 self.__getitem__ = lambda name: 1
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
20
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
21 def __getitem__(self, name):
5381
0942fe89e82e Python 3 preparation: change "x.has_key(y)" to "y in x".
Joseph Myers <jsm@polyomino.org.uk>
parents: 5376
diff changeset
22 return name in self.keys
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
23
3019
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
24 def ensureParentsExist(dest):
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
25 if not os.path.exists(os.path.dirname(dest)):
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
26 os.makedirs(os.path.dirname(dest))
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
27
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
28 class PrioList:
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
29 '''Manages a sorted list.
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
30
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
31 Currently only implements method 'append' and iteration from a
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
32 full list interface.
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
33 Implementation: We manage a "sorted" status and sort on demand.
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
34 Appending to the list will require re-sorting before use.
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
35 >>> p = PrioList()
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
36 >>> for i in 5,7,1,-1:
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
37 ... p.append(i)
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
38 ...
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
39 >>> for k in p:
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
40 ... print k
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
41 ...
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
42 -1
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
43 1
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
44 5
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
45 7
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
46
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
47 '''
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
48 def __init__(self):
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
49 self.list = []
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
50 self.sorted = True
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
51
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
52 def append(self, item):
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
53 self.list.append(item)
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
54 self.sorted = False
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
55
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
56 def __iter__(self):
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
57 if not self.sorted:
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
58 self.list.sort()
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
59 self.sorted = True
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
60 return iter(self.list)
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
61
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
62 class Progress:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
63 '''Progress display for console applications.
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
64
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
65 See __main__ block at end of file for sample usage.
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
66 '''
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
67 def __init__(self, info, sequence):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
68 self.info = info
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
69 self.sequence = iter(sequence)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
70 self.total = len(sequence)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
71 self.start = self.now = time.time()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
72 self.num = 0
5400
2120f77554d5 Python 3 preparation: use // and __truediv__ as needed.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5396
diff changeset
73 self.stepsize = self.total // 100 or 1
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
74 self.steptimes = []
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
75 self.display()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
76
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
77 def __iter__(self): return self
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
78
5396
831787cf6694 Python 3 preparation: update next() usage for iterators.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
79 def __next__(self):
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
80 self.num += 1
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
81
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
82 if self.num > self.total:
5376
64b05e24dbd8 Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5043
diff changeset
83 print(self.info, 'done', ' '*(75-len(self.info)-6))
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
84 sys.stdout.flush()
5396
831787cf6694 Python 3 preparation: update next() usage for iterators.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
85 return next(self.sequence)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
86
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
87 if self.num % self.stepsize:
5396
831787cf6694 Python 3 preparation: update next() usage for iterators.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
88 return next(self.sequence)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
89
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
90 self.display()
5396
831787cf6694 Python 3 preparation: update next() usage for iterators.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
91 return next(self.sequence)
831787cf6694 Python 3 preparation: update next() usage for iterators.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
92 # Python 2 compatibility:
831787cf6694 Python 3 preparation: update next() usage for iterators.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
93 next = __next__
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
94
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
95 def display(self):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
96 # figure how long we've spent - guess how long to go
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
97 now = time.time()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
98 steptime = now - self.now
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
99 self.steptimes.insert(0, steptime)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
100 if len(self.steptimes) > 5:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
101 self.steptimes.pop()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
102 steptime = sum(self.steptimes) / len(self.steptimes)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
103 self.now = now
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
104 eta = steptime * ((self.total - self.num)/self.stepsize)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
105
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
106 # tell it like it is (or might be)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
107 if now - self.start > 3:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
108 M = eta / 60
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
109 H = M / 60
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
110 M = M % 60
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
111 S = eta % 60
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
112 if self.total:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
113 s = '%s %2d%% (ETA %02d:%02d:%02d)'%(self.info,
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
114 self.num * 100. / self.total, H, M, S)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
115 else:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
116 s = '%s 0%% (ETA %02d:%02d:%02d)'%(self.info, H, M, S)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
117 elif self.total:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
118 s = '%s %2d%%'%(self.info, self.num * 100. / self.total)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
119 else:
3589
1be293265e61 woo, that was quick
Richard Jones <richard@users.sourceforge.net>
parents: 3544
diff changeset
120 s = '%s %d done'%(self.info, self.num)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
121 sys.stdout.write(s + ' '*(75-len(s)) + '\r')
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
122 sys.stdout.flush()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
123
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
124 # vim: set et sts=4 sw=4 :
Roundup Issue Tracker: http://roundup-tracker.org/