annotate test/wsgi_liveserver.py @ 8446:14c7c07b32d8

feature: add thread local trace_id and trace_reason to logging. Added trace_id to default logging so that all logs for a given request share the same trace_id. This allows correlation of logs across a request. admin_guide.txt, upgrading.txt: add docs update sample configs to include trace_id. rewrite logging docs in admin_guide. Hopefully they are clearer now. clean up some stuff in the logging config file docs. admin.py: add decorators to run_command to enable trace_id. change calls to db.commit() to use run_command to get trace_id. configuration.py: clean up imports. update docstrings, comments and inline docs. add trace_id to default log format. add function for testing decorated with trace_id. add support for dumping stack trace in logging. add check for pytest in sys.modules to enable log propagation when pytest is running. Otherwise tests fail as the caplog logger doesn't see the roundup logs. logcontext.py: new file to handle thread local contextvar mangement. mailgw.py: add decorators for trace_id etc. scripts/roundup_xlmrpc_server.py: add decorators for trace_id etc. fix encoding bug turning bytes into a string. fix command line issue where we can't set encoding. (not sure if changing encoding via command line even works) cgi/client.py decorate two entry points for trace_id etc. cgi/wsgi_handler.py: decorate entry point for trace_id etc. test/test_config.py: add test for trace_id in new log format. test various cases for sinfo and errors in formating msg.
author John Rouillard <rouilj@ieee.org>
date Tue, 16 Sep 2025 22:53:00 -0400
parents 3630246c5c36
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6383
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
2 """
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
3 wsgi-liveserver provides a simple LiverServerTestCase class that can be used to
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
4 help start a web server in the background to serve a WSGI compliant application
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
5 for use with testing. Generally it will be used in conjuction with something
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
6 like Selenium to perform a series of functional tests using a browser.
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
7
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
8 Licensed under the GNU GPL v3
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
9
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
10 Copyright (c) 2013 John Kristensen (unless explicitly stated otherwise).
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
11 """
8248
f6923d2ba9a5 test: issue2551366. Probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 6383
diff changeset
12 import errno
f6923d2ba9a5 test: issue2551366. Probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 6383
diff changeset
13 import socket
6383
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
14 import threading
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
15 import unittest
8257
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
16 from wsgiref.simple_server import WSGIRequestHandler, make_server
6383
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
17
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
18 __author__ = 'John Kristensen'
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
19 __version__ = '0.3.1'
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
20 __license__ = 'GPLv3'
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
21
8250
c7f63e9cd131 test: add comment that I modified wsgi_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8249
diff changeset
22 """
c7f63e9cd131 test: add comment that I modified wsgi_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8249
diff changeset
23 Classmethod probe_ports() added by John Rouillard 2024.
c7f63e9cd131 test: add comment that I modified wsgi_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8249
diff changeset
24 """
6383
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
25
8257
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
26
6383
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
27 class QuietHandler(WSGIRequestHandler):
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
28 def log_request(*args, **kwargs):
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
29 pass
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
30
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
31
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
32 class LiveServerTestCase(unittest.TestCase):
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
33
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
34 port_range = (8080, 8090)
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
35
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
36 def create_app(self):
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
37 """Create your wsgi app and return it."""
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
38 raise NotImplementedError
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
39
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
40 def __call__(self, result=None):
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
41 """
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
42 Do some custom setup stuff and then hand off to TestCase to do its
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
43 thing.
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
44 """
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
45 try:
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
46 self._pre_setup()
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
47 super(LiveServerTestCase, self).__call__(result)
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
48 finally:
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
49 self._post_teardown()
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
50
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
51 def url_base(self):
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
52 """Return the url of the test server."""
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
53 return 'http://{0}:{1}'.format(self.host, self.port)
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
54
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
55 def _pre_setup(self):
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
56 """Setup and start the test server in the background."""
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
57 self._server = None
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
58
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
59 self.host = 'localhost'
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
60 self.port = self.port_range[0]
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
61 self._thread = None
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
62
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
63 # Get the app
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
64 self.app = self.create_app()
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
65
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
66 # Cycle through the port range to find a free port
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
67 while self._server is None and self.port <= self.port_range[1]:
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
68 try:
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
69 self._server = make_server(self.host, self.port, self.app,
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
70 handler_class=QuietHandler)
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
71 except socket.error:
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
72 self.port += 1
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
73
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
74 # No free port, raise an exception
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
75 if self._server is None:
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
76 raise socket.error('Ports {0}-{1} are all already in use'.format(
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
77 *self.port_range))
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
78
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
79 # Start the test server in the background
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
80 self._thread = threading.Thread(target=self._server.serve_forever)
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
81 self._thread.start()
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
82
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
83 def _post_teardown(self):
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
84 """Stop the test server."""
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
85 if self._thread is not None:
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
86 self._server.shutdown()
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
87 self._server.server_close()
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
88 self._thread.join()
e9760702bf0c Add live server test to suite.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
89 del self._server
8248
f6923d2ba9a5 test: issue2551366. Probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 6383
diff changeset
90
8249
d317a41dfcf3 test: make probe_ports a class method under python2.
John Rouillard <rouilj@ieee.org>
parents: 8248
diff changeset
91 @classmethod
d317a41dfcf3 test: make probe_ports a class method under python2.
John Rouillard <rouilj@ieee.org>
parents: 8248
diff changeset
92 def probe_ports(cls, start=port_range[0], end=port_range[1]):
8257
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
93
8248
f6923d2ba9a5 test: issue2551366. Probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 6383
diff changeset
94 port = start
8257
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
95
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
96 while port <= end:
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
97 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
98 s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
99
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
100 try:
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
101 s.connect(('127.0.0.1', port))
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
102 except socket.error as e:
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
103 if not hasattr(e, 'args') or e.args[0] != errno.ECONNREFUSED:
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
104 raise
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
105 return port
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
106 else:
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
107 s.close()
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
108 port += 1
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
109
3630246c5c36 test: issue2551366. fix probe for open port in test_liveserver.py
John Rouillard <rouilj@ieee.org>
parents: 8250
diff changeset
110 return None

Roundup Issue Tracker: http://roundup-tracker.org/