Mercurial > p > roundup > code

annotate roundup/support.py @ 8540:e8d1da6e3571

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
bug: fix traceback in roundup-admin init with bad config values initialize accepts setting values for config.ini file settings. If they are not valid, you got a python traceback. ConfigurationError exceptions are now trapped. The admin.py's usageError_feedback method is used to inform the user. Also the feedback message now starts with a newline making it easier to read by separating it from command that caused the issue.
author John Rouillard <rouilj@ieee.org>
date Mon, 23 Mar 2026 13:18:41 -0400
parents 224ccb8b49ca
children 9c3ec0a5c7fc
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
1 """Implements various support classes and functions used in a number of
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
2 places in Roundup code.
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
3 """
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
4
5376
64b05e24dbd8 Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5043
diff changeset
5 from __future__ import print_function
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
6 __docformat__ = 'restructuredtext'
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
7
6018
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
8 import os, time, sys
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
9
3019
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
10
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
11 class TruthDict:
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
12 '''Returns True for valid keys, False for others.
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
13 '''
8472
224ccb8b49ca refactor: change some classes to use __slots__
John Rouillard <rouilj@ieee.org>
parents: 7846
diff changeset
14
224ccb8b49ca refactor: change some classes to use __slots__
John Rouillard <rouilj@ieee.org>
parents: 7846
diff changeset
15 __slots__ = ('keys',)
224ccb8b49ca refactor: change some classes to use __slots__
John Rouillard <rouilj@ieee.org>
parents: 7846
diff changeset
16
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
17 def __init__(self, keys):
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
18 if keys:
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
19 self.keys = {}
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
20 for col in keys:
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
21 self.keys[col] = 1
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
22
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
23 def __getitem__(self, name):
5442
afd9fd3a0edb Python 3 preparation: avoid assigning to instance __getitem__ in TruthDict.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5400
diff changeset
24 if hasattr(self, 'keys'):
afd9fd3a0edb Python 3 preparation: avoid assigning to instance __getitem__ in TruthDict.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5400
diff changeset
25 return name in self.keys
afd9fd3a0edb Python 3 preparation: avoid assigning to instance __getitem__ in TruthDict.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5400
diff changeset
26 else:
afd9fd3a0edb Python 3 preparation: avoid assigning to instance __getitem__ in TruthDict.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5400
diff changeset
27 return True
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
28
6018
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
29
3019
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
30 def ensureParentsExist(dest):
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
31 if not os.path.exists(os.path.dirname(dest)):
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
32 os.makedirs(os.path.dirname(dest))
293a17149765 First cut at exporting/importing file content from and to the hyperdb.
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents: 2984
diff changeset
33
6018
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
34
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
35 class PrioList:
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
36 '''Manages a sorted list.
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
37
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
38 Currently only implements method 'append' and iteration from a
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
39 full list interface.
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
40 Implementation: We manage a "sorted" status and sort on demand.
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
41 Appending to the list will require re-sorting before use.
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
42 >>> p = PrioList()
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
43 >>> for i in 5,7,1,-1:
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
44 ... p.append(i)
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
45 ...
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
46 >>> for k in p:
7114
33eb82ad26ba issue2551250: Fix sorting of detectors
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6018
diff changeset
47 ... print (k)
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
48 ...
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
49 -1
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
50 1
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
51 5
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
52 7
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
53
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
54 '''
8472
224ccb8b49ca refactor: change some classes to use __slots__
John Rouillard <rouilj@ieee.org>
parents: 7846
diff changeset
55
224ccb8b49ca refactor: change some classes to use __slots__
John Rouillard <rouilj@ieee.org>
parents: 7846
diff changeset
56 __slots__ = ('key', 'list', 'sorted')
224ccb8b49ca refactor: change some classes to use __slots__
John Rouillard <rouilj@ieee.org>
parents: 7846
diff changeset
57
7114
33eb82ad26ba issue2551250: Fix sorting of detectors
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6018
diff changeset
58 def __init__(self, key=None):
6018
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
59 self.list = []
7114
33eb82ad26ba issue2551250: Fix sorting of detectors
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6018
diff changeset
60 self.key = key
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
61 self.sorted = True
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
62
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
63 def append(self, item):
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
64 self.list.append(item)
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
65 self.sorted = False
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
66
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
67 def __iter__(self):
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
68 if not self.sorted:
7114
33eb82ad26ba issue2551250: Fix sorting of detectors
Ralf Schlatterbeck <rsc@runtux.com>
parents: 6018
diff changeset
69 self.list.sort(key=self.key)
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
70 self.sorted = True
3635
53987aa153d2 Transitive-property support.
Ralf Schlatterbeck <schlatterbeck@users.sourceforge.net>
parents: 3634
diff changeset
71 return iter(self.list)
3487
a2ae11191968 added class PrioList (patch from rfe [SF#413165])
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 3019
diff changeset
72
6018
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
73
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
74 class Progress:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
75 '''Progress display for console applications.
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
76
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
77 '''
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
78 def __init__(self, info, sequence):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
79 self.info = info
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
80 self.sequence = iter(sequence)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
81 self.total = len(sequence)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
82 self.start = self.now = time.time()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
83 self.num = 0
5400
2120f77554d5 Python 3 preparation: use // and __truediv__ as needed.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5396
diff changeset
84 self.stepsize = self.total // 100 or 1
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
85 self.steptimes = []
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
86 self.display()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
87
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
88 def __iter__(self): return self
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
89
5396
831787cf6694 Python 3 preparation: update next() usage for iterators.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
90 def __next__(self):
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
91 self.num += 1
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
92
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
93 if self.num > self.total:
5376
64b05e24dbd8 Python 3 preparation: convert print to a function.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5043
diff changeset
94 print(self.info, 'done', ' '*(75-len(self.info)-6))
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
95 sys.stdout.flush()
5396
831787cf6694 Python 3 preparation: update next() usage for iterators.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
96 return next(self.sequence)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
97
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
98 if self.num % self.stepsize:
5396
831787cf6694 Python 3 preparation: update next() usage for iterators.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
99 return next(self.sequence)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
100
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
101 self.display()
5396
831787cf6694 Python 3 preparation: update next() usage for iterators.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
102 return next(self.sequence)
831787cf6694 Python 3 preparation: update next() usage for iterators.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
103 # Python 2 compatibility:
831787cf6694 Python 3 preparation: update next() usage for iterators.
Joseph Myers <jsm@polyomino.org.uk>
parents: 5381
diff changeset
104 next = __next__
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
105
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
106 def display(self):
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
107 # figure how long we've spent - guess how long to go
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
108 now = time.time()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
109 steptime = now - self.now
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
110 self.steptimes.insert(0, steptime)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
111 if len(self.steptimes) > 5:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
112 self.steptimes.pop()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
113 steptime = sum(self.steptimes) / len(self.steptimes)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
114 self.now = now
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
115 eta = steptime * ((self.total - self.num)/self.stepsize)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
116
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
117 # tell it like it is (or might be)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
118 if now - self.start > 3:
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
119 M = eta / 60
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
120 H = M / 60
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
121 M = M % 60
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
122 S = eta % 60
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
123 if self.total:
6018
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
124 s = '%s %2d%% (ETA %02d:%02d:%02d)' % (self.info,
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
125 self.num * 100. / self.total, H, M, S)
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
126 else:
6018
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
127 s = '%s 0%% (ETA %02d:%02d:%02d)' % (self.info, H, M, S)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
128 elif self.total:
6018
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
129 s = '%s %2d%%' % (self.info, self.num * 100. / self.total)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
130 else:
6018
01643d37785f flake8 cleanup: remove unused re import; whitespace normalization.
John Rouillard <rouilj@ieee.org>
parents: 5442
diff changeset
131 s = '%s %d done' % (self.info, self.num)
3544
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
132 sys.stdout.write(s + ' '*(75-len(s)) + '\r')
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
133 sys.stdout.flush()
5cd1c83dea50 Features and fixes.
Richard Jones <richard@users.sourceforge.net>
parents: 3487
diff changeset
134
2984
887d631ea2ba general use
Richard Jones <richard@users.sourceforge.net>
parents:
diff changeset
135 # vim: set et sts=4 sw=4 :
Roundup Issue Tracker: http://roundup-tracker.org/