annotate roundup/cgi/exceptions.py @ 6565:2c2dbfc332ba

Try to handle multiple connections better. The session database is a hot spot. When multiple requests (e.g. 20) come in at the same time session database contention can get great. The original code didn't retry session database access when the open failed. This resulted in errors at the client. The second pass delayed 0.01 seconds and retried. It was better but we still had multiple second stalls. I think the first request got in, everybody else backed up and then retried at the same time. Again they stepped on each other. With logging I would see many counters go all the way to low single digits or to -1 indicating falure. This pass uses randomint to generate delays from 0-.125 seconds in 5ms increments. This performs better in testing. I rarely saw a counter less than 13 (2 failed retries). Current logging starts after 6 failures and counts down until success or failure.
author John Rouillard <rouilj@ieee.org>
date Thu, 16 Dec 2021 20:02:00 -0500
parents c177e7128dc9
children 91ab3e0ffcd0
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4083
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents: 2924
diff changeset
1 """Exceptions for use in Roundup's web interface.
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents: 2924
diff changeset
2 """
2129
3fd672293712 add and use Reject exception [SF#700265]
Richard Jones <richard@users.sourceforge.net>
parents: 2062
diff changeset
3
3fd672293712 add and use Reject exception [SF#700265]
Richard Jones <richard@users.sourceforge.net>
parents: 2062
diff changeset
4 __docformat__ = 'restructuredtext'
3fd672293712 add and use Reject exception [SF#700265]
Richard Jones <richard@users.sourceforge.net>
parents: 2062
diff changeset
5
6039
0dc1e0222353 flake8 whitespace changes, suppress unused import warning
John Rouillard <rouilj@ieee.org>
parents: 5877
diff changeset
6 from roundup.exceptions import LoginError, Unauthorised # noqa: F401
5800
1a835db41674 Call cgi.escape only on python 2. Replace with html.escapeif it can be
John Rouillard <rouilj@ieee.org>
parents: 5630
diff changeset
7
5837
883c9e90b403 Fix problem with cgi.escape being depricated a different way. This way
John Rouillard <rouilj@ieee.org>
parents: 5800
diff changeset
8 from roundup.anypy.html import html_escape
2062
f6d7ccce8d96 cgi exceptions fix
Richard Jones <richard@users.sourceforge.net>
parents: 2052
diff changeset
9
6123
c177e7128dc9 issue2551083 Replace BaseException and Exception with RoundupException
John Rouillard <rouilj@ieee.org>
parents: 6039
diff changeset
10 from roundup.exceptions import RoundupException
6039
0dc1e0222353 flake8 whitespace changes, suppress unused import warning
John Rouillard <rouilj@ieee.org>
parents: 5877
diff changeset
11
6123
c177e7128dc9 issue2551083 Replace BaseException and Exception with RoundupException
John Rouillard <rouilj@ieee.org>
parents: 6039
diff changeset
12 class RoundupCGIException(RoundupException):
c177e7128dc9 issue2551083 Replace BaseException and Exception with RoundupException
John Rouillard <rouilj@ieee.org>
parents: 6039
diff changeset
13 pass
c177e7128dc9 issue2551083 Replace BaseException and Exception with RoundupException
John Rouillard <rouilj@ieee.org>
parents: 6039
diff changeset
14
c177e7128dc9 issue2551083 Replace BaseException and Exception with RoundupException
John Rouillard <rouilj@ieee.org>
parents: 6039
diff changeset
15 class HTTPException(RoundupCGIException):
2004
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
16 pass
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
17
6039
0dc1e0222353 flake8 whitespace changes, suppress unused import warning
John Rouillard <rouilj@ieee.org>
parents: 5877
diff changeset
18
2004
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
19 class Redirect(HTTPException):
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
20 pass
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
21
6039
0dc1e0222353 flake8 whitespace changes, suppress unused import warning
John Rouillard <rouilj@ieee.org>
parents: 5877
diff changeset
22
2004
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
23 class NotFound(HTTPException):
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
24 pass
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
25
6039
0dc1e0222353 flake8 whitespace changes, suppress unused import warning
John Rouillard <rouilj@ieee.org>
parents: 5877
diff changeset
26
2004
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
27 class NotModified(HTTPException):
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
28 pass
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
29
6039
0dc1e0222353 flake8 whitespace changes, suppress unused import warning
John Rouillard <rouilj@ieee.org>
parents: 5877
diff changeset
30
5630
07abc8d36940 Add etag support to rest interface to prevent multiple users from
John Rouillard <rouilj@ieee.org>
parents: 5264
diff changeset
31 class PreconditionFailed(HTTPException):
07abc8d36940 Add etag support to rest interface to prevent multiple users from
John Rouillard <rouilj@ieee.org>
parents: 5264
diff changeset
32 pass
07abc8d36940 Add etag support to rest interface to prevent multiple users from
John Rouillard <rouilj@ieee.org>
parents: 5264
diff changeset
33
6039
0dc1e0222353 flake8 whitespace changes, suppress unused import warning
John Rouillard <rouilj@ieee.org>
parents: 5877
diff changeset
34
6123
c177e7128dc9 issue2551083 Replace BaseException and Exception with RoundupException
John Rouillard <rouilj@ieee.org>
parents: 6039
diff changeset
35 class DetectorError(RoundupException):
5079
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 4083
diff changeset
36 """Raised when a detector throws an exception.
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 4083
diff changeset
37 Contains details of the exception."""
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 4083
diff changeset
38 def __init__(self, subject, html, txt):
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 4083
diff changeset
39 self.subject = subject
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 4083
diff changeset
40 self.html = html
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 4083
diff changeset
41 self.txt = txt
6039
0dc1e0222353 flake8 whitespace changes, suppress unused import warning
John Rouillard <rouilj@ieee.org>
parents: 5877
diff changeset
42 BaseException.__init__(self, subject + ' ' + txt)
0dc1e0222353 flake8 whitespace changes, suppress unused import warning
John Rouillard <rouilj@ieee.org>
parents: 5877
diff changeset
43
5079
65fef7858606 issue2550826 IOError in detector causes apache 'premature end of script headers' error
John Rouillard <rouilj@ieee.org>
parents: 4083
diff changeset
44
2004
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
45 class FormError(ValueError):
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
46 """An 'expected' exception occurred during form parsing.
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
47
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
48 That is, something we know can go wrong, and don't want to alarm the user
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
49 with.
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
50
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
51 We trap this at the user interface level and feed back a nice error to the
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
52 user.
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
53
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
54 """
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
55 pass
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
56
6039
0dc1e0222353 flake8 whitespace changes, suppress unused import warning
John Rouillard <rouilj@ieee.org>
parents: 5877
diff changeset
57
6123
c177e7128dc9 issue2551083 Replace BaseException and Exception with RoundupException
John Rouillard <rouilj@ieee.org>
parents: 6039
diff changeset
58 class SendFile(RoundupException):
2004
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
59 """Send a file from the database."""
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
60
6039
0dc1e0222353 flake8 whitespace changes, suppress unused import warning
John Rouillard <rouilj@ieee.org>
parents: 5877
diff changeset
61
6123
c177e7128dc9 issue2551083 Replace BaseException and Exception with RoundupException
John Rouillard <rouilj@ieee.org>
parents: 6039
diff changeset
62 class SendStaticFile(RoundupException):
2004
1782fe36e7b8 Move out parts of client.py to new modules:
Johannes Gijsbers <jlgijsbers@users.sourceforge.net>
parents:
diff changeset
63 """Send a static file from the instance html directory."""
2052
78e6a1e4984e forward-port from maint branch
Richard Jones <richard@users.sourceforge.net>
parents: 2004
diff changeset
64
6039
0dc1e0222353 flake8 whitespace changes, suppress unused import warning
John Rouillard <rouilj@ieee.org>
parents: 5877
diff changeset
65
6123
c177e7128dc9 issue2551083 Replace BaseException and Exception with RoundupException
John Rouillard <rouilj@ieee.org>
parents: 6039
diff changeset
66 class SeriousError(RoundupException):
2052
78e6a1e4984e forward-port from maint branch
Richard Jones <richard@users.sourceforge.net>
parents: 2004
diff changeset
67 """Raised when we can't reasonably display an error message on a
78e6a1e4984e forward-port from maint branch
Richard Jones <richard@users.sourceforge.net>
parents: 2004
diff changeset
68 templated page.
78e6a1e4984e forward-port from maint branch
Richard Jones <richard@users.sourceforge.net>
parents: 2004
diff changeset
69
78e6a1e4984e forward-port from maint branch
Richard Jones <richard@users.sourceforge.net>
parents: 2004
diff changeset
70 The exception value will be displayed in the error page, HTML
78e6a1e4984e forward-port from maint branch
Richard Jones <richard@users.sourceforge.net>
parents: 2004
diff changeset
71 escaped.
78e6a1e4984e forward-port from maint branch
Richard Jones <richard@users.sourceforge.net>
parents: 2004
diff changeset
72 """
78e6a1e4984e forward-port from maint branch
Richard Jones <richard@users.sourceforge.net>
parents: 2004
diff changeset
73 def __str__(self):
4083
bbab97f8ffb2 XMLRPC improvements:
Stefan Seefeld <stefan@seefeld.name>
parents: 2924
diff changeset
74 return """
2052
78e6a1e4984e forward-port from maint branch
Richard Jones <richard@users.sourceforge.net>
parents: 2004
diff changeset
75 <html><head><title>Roundup issue tracker: An error has occurred</title>
2278
93bd8c4d43ef in HTML produced by SeriousError.__str__():
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2129
diff changeset
76 <link rel="stylesheet" type="text/css" href="@@file/style.css">
2052
78e6a1e4984e forward-port from maint branch
Richard Jones <richard@users.sourceforge.net>
parents: 2004
diff changeset
77 </head>
78e6a1e4984e forward-port from maint branch
Richard Jones <richard@users.sourceforge.net>
parents: 2004
diff changeset
78 <body class="body" marginwidth="0" marginheight="0">
78e6a1e4984e forward-port from maint branch
Richard Jones <richard@users.sourceforge.net>
parents: 2004
diff changeset
79 <p class="error-message">%s</p>
78e6a1e4984e forward-port from maint branch
Richard Jones <richard@users.sourceforge.net>
parents: 2004
diff changeset
80 </body></html>
6039
0dc1e0222353 flake8 whitespace changes, suppress unused import warning
John Rouillard <rouilj@ieee.org>
parents: 5877
diff changeset
81 """ % html_escape(self.args[0])
2052
78e6a1e4984e forward-port from maint branch
Richard Jones <richard@users.sourceforge.net>
parents: 2004
diff changeset
82
2924
df4a3355ee8f added LoginError; fix vim modeline
Alexander Smishlajev <a1s@users.sourceforge.net>
parents: 2278
diff changeset
83 # vim: set filetype=python sts=4 sw=4 et si :

Roundup Issue Tracker: http://roundup-tracker.org/