Mercurial > p > roundup > code

annotate roundup/support.py @ 7531:913a73b9fab5 2.3.0

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Update for 2.3.0 release
author John Rouillard <rouilj@ieee.org>
date Wed, 12 Jul 2023 23:00:25 -0400
parents 33eb82ad26ba
children 8ef97f7cfb6d
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1 """Implements various support classes and functions used in a number of
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2 places in Roundup code.
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3 """
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
4
5376
64b05e24dbd8 Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5043
diff changeset
5 from __future__ import print_function
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
6 __docformat__ = 'restructuredtext'
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
7
6018
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
8 import os, time, sys
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
9
3019
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
10
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
11 class TruthDict:
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
12 '''Returns True for valid keys, False for others.
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
13 '''
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
14 def __init__(self, keys):
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
15 if keys:
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
16 self.keys = {}
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
17 for col in keys:
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
18 self.keys[col] = 1
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
19
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
20 def __getitem__(self, name):
5442
afd9fd3a0edb Python 3 preparation: avoid assigning to instance __getitem__ in TruthDict.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5400
diff changeset
21 if hasattr(self, 'keys'):
afd9fd3a0edb Python 3 preparation: avoid assigning to instance __getitem__ in TruthDict.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5400
diff changeset
22 return name in self.keys
afd9fd3a0edb Python 3 preparation: avoid assigning to instance __getitem__ in TruthDict.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5400
diff changeset
23 else:
afd9fd3a0edb Python 3 preparation: avoid assigning to instance __getitem__ in TruthDict.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5400
diff changeset
24 return True
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
25
6018
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
26
3019
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
27 def ensureParentsExist(dest):
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
28 if not os.path.exists(os.path.dirname(dest)):
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
29 os.makedirs(os.path.dirname(dest))
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
30
6018
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
31
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
32 class PrioList:
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
33 '''Manages a sorted list.
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
34
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
35 Currently only implements method 'append' and iteration from a
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
36 full list interface.
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
37 Implementation: We manage a "sorted" status and sort on demand.
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
38 Appending to the list will require re-sorting before use.
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
39 >>> p = PrioList()
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
40 >>> for i in 5,7,1,-1:
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
41 ... p.append(i)
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
42 ...
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
43 >>> for k in p:
7114
33eb82ad26ba issue2551250: Fix sorting of detectors
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6018
diff changeset
44 ... print (k)
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
45 ...
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
46 -1
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
47 1
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
48 5
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
49 7
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
50
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
51 '''
7114
33eb82ad26ba issue2551250: Fix sorting of detectors
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6018
diff changeset
52 def __init__(self, key=None):
6018
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
53 self.list = []
7114
33eb82ad26ba issue2551250: Fix sorting of detectors
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6018
diff changeset
54 self.key = key
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
55 self.sorted = True
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
56
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
57 def append(self, item):
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
58 self.list.append(item)
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
59 self.sorted = False
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
60
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
61 def __iter__(self):
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
62 if not self.sorted:
7114
33eb82ad26ba issue2551250: Fix sorting of detectors
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6018
diff changeset
63 self.list.sort(key=self.key)
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
64 self.sorted = True
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
65 return iter(self.list)
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
66
6018
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
67
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
68 class Progress:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
69 '''Progress display for console applications.
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
70
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
71 See __main__ block at end of file for sample usage.
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
72 '''
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
73 def __init__(self, info, sequence):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
74 self.info = info
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
75 self.sequence = iter(sequence)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
76 self.total = len(sequence)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
77 self.start = self.now = time.time()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
78 self.num = 0
5400
2120f77554d5 Python 3 preparation: use // and __truediv__ as needed.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5396
diff changeset
79 self.stepsize = self.total // 100 or 1
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
80 self.steptimes = []
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
81 self.display()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
82
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
83 def __iter__(self): return self
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
84
5396
831787cf6694 Python 3 preparation: update next() usage for iterators.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
85 def __next__(self):
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
86 self.num += 1
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
87
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
88 if self.num > self.total:
5376
64b05e24dbd8 Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5043
diff changeset
89 print(self.info, 'done', ' '*(75-len(self.info)-6))
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
90 sys.stdout.flush()
5396
831787cf6694 Python 3 preparation: update next() usage for iterators.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
91 return next(self.sequence)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
92
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
93 if self.num % self.stepsize:
5396
831787cf6694 Python 3 preparation: update next() usage for iterators.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
94 return next(self.sequence)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
95
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
96 self.display()
5396
831787cf6694 Python 3 preparation: update next() usage for iterators.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
97 return next(self.sequence)
831787cf6694 Python 3 preparation: update next() usage for iterators.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
98 # Python 2 compatibility:
831787cf6694 Python 3 preparation: update next() usage for iterators.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
99 next = __next__
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
100
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
101 def display(self):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
102 # figure how long we've spent - guess how long to go
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
103 now = time.time()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
104 steptime = now - self.now
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
105 self.steptimes.insert(0, steptime)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
106 if len(self.steptimes) > 5:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
107 self.steptimes.pop()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
108 steptime = sum(self.steptimes) / len(self.steptimes)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
109 self.now = now
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
110 eta = steptime * ((self.total - self.num)/self.stepsize)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
111
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
112 # tell it like it is (or might be)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
113 if now - self.start > 3:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
114 M = eta / 60
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
115 H = M / 60
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
116 M = M % 60
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
117 S = eta % 60
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
118 if self.total:
6018
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
119 s = '%s %2d%% (ETA %02d:%02d:%02d)' % (self.info,
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
120 self.num * 100. / self.total, H, M, S)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
121 else:
6018
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
122 s = '%s 0%% (ETA %02d:%02d:%02d)' % (self.info, H, M, S)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
123 elif self.total:
6018
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
124 s = '%s %2d%%' % (self.info, self.num * 100. / self.total)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
125 else:
6018
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
126 s = '%s %d done' % (self.info, self.num)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
127 sys.stdout.write(s + ' '*(75-len(s)) + '\r')
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
128 sys.stdout.flush()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
129
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
130 # vim: set et sts=4 sw=4 :
Roundup Issue Tracker: http://roundup-tracker.org/