Mercurial > p > roundup > code

annotate roundup/logcontext.py @ 8446:14c7c07b32d8

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
feature: add thread local trace_id and trace_reason to logging. Added trace_id to default logging so that all logs for a given request share the same trace_id. This allows correlation of logs across a request. admin_guide.txt, upgrading.txt: add docs update sample configs to include trace_id. rewrite logging docs in admin_guide. Hopefully they are clearer now. clean up some stuff in the logging config file docs. admin.py: add decorators to run_command to enable trace_id. change calls to db.commit() to use run_command to get trace_id. configuration.py: clean up imports. update docstrings, comments and inline docs. add trace_id to default log format. add function for testing decorated with trace_id. add support for dumping stack trace in logging. add check for pytest in sys.modules to enable log propagation when pytest is running. Otherwise tests fail as the caplog logger doesn't see the roundup logs. logcontext.py: new file to handle thread local contextvar mangement. mailgw.py: add decorators for trace_id etc. scripts/roundup_xlmrpc_server.py: add decorators for trace_id etc. fix encoding bug turning bytes into a string. fix command line issue where we can't set encoding. (not sure if changing encoding via command line even works) cgi/client.py decorate two entry points for trace_id etc. cgi/wsgi_handler.py: decorate entry point for trace_id etc. test/test_config.py: add test for trace_id in new log format. test various cases for sinfo and errors in formating msg.
author John Rouillard <rouilj@ieee.org>
date Tue, 16 Sep 2025 22:53:00 -0400
parents
children 3df6b8988074
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
8446
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
1 import contextvars
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
2 import functools
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
3 import logging
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
4 import os
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
5 import uuid
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
6
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
7 logger = logging.getLogger("roundup.logcontext")
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
8
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
9
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
10 class SimpleSentinel:
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
11 """A hack to get a sentinel value where I can define __str__().
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
12
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
13 I was using sentinel = object(). However some code paths
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
14 resulted in the sentinel object showing up as an argument
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
15 to print or logging.warning|error|debug(...). In this case
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
16 seeing "<class 'object'>" in the output isn't useful.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
17
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
18 So I created this class (with slots) as a fast method where
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
19 I could control the __str__ representation.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
20
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
21 """
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
22 __slots__ = ("name", "str")
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
23
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
24 def __init__(self, name=None, str_value=""):
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
25 self.name = name
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
26 self.str = str_value
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
27
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
28 def __str__(self):
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
29 # Generate a string without whitespace.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
30 # Used in logging where whitespace could be
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
31 # a field delimiter
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
32 return ("%s%s" % (
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
33 self.name + "-" if self.name else "",
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
34 self.str)).replace(" ", "_")
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
35
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
36 def __repr__(self):
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
37 return 'SimpleSentinel(name=%s, str_value="%s")' % (
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
38 self.name, self.str)
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
39
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
40
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
41 # store the context variable names in a dict. Because
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
42 # contactvars.copy_context().items() returns nothing if set has
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
43 # not been called on a context var. I need the contextvar names
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
44 # even if they have not been set.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
45 ctx_vars = {}
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
46
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
47 # set up sentinel values that will print a suitable error value
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
48 # and the context vars they are associated with.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
49 _SENTINEL_ID = SimpleSentinel("trace_id", "not set")
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
50 ctx_vars['trace_id'] = contextvars.ContextVar("trace_id", default=_SENTINEL_ID)
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
51
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
52
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
53 _SENTINEL_REASON = SimpleSentinel("trace_reason", "missing")
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
54 ctx_vars['trace_reason'] = contextvars.ContextVar("trace_reason",
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
55 default=_SENTINEL_REASON)
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
56
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
57
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
58 def shorten_int_uuid(uuid):
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
59 """Encode a UUID integer in a shorter form for display.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
60
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
61 A uuid is long. Make a shorter version that takes less room
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
62 in a log line.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
63 """
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
64
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
65 alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz1234567890"
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
66 result = ""
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
67 while uuid:
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
68 uuid, t = divmod(uuid, len(alphabet))
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
69 result += alphabet[t]
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
70 return result or "0"
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
71
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
72
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
73 def gen_trace_id():
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
74 """Decorator to generate a trace id (encoded uuid4) and add to contextvar
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
75
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
76 The logging routine uses this to label every log line. All
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
77 logs with the same trace_id should be generated from a
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
78 single request.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
79
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
80 This decorator is applied to an entry point for a request.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
81 Different methods of invoking Roundup have different entry
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
82 points. As a result, this decorator can be called multiple
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
83 times as some entry points can traverse another entry point
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
84 used by a different invocation method. It will not set a
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
85 trace_id if one is already assigned.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
86
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
87 A uuid4() is used as the uuid, but to shorten the log line,
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
88 the uuid4 integer is encoded into a 62 character ascii
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
89 alphabet (A-Za-z0-9).
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
90
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
91 This decorator may produce duplicate (colliding) trace_id's
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
92 when used with multiple processes on some platforms where
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
93 uuid.uuid4().is_safe is unknown. Probability of a collision
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
94 is unknown.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
95
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
96 """
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
97 def decorator(func):
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
98 @functools.wraps(func)
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
99 def wrapper(*args, **kwargs):
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
100 prev = None
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
101 trace_id = ctx_vars['trace_id']
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
102 if trace_id.get() is _SENTINEL_ID:
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
103 prev = trace_id.set(shorten_int_uuid(uuid.uuid4().int))
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
104 try:
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
105 r = func(*args, **kwargs)
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
106 finally:
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
107 if prev:
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
108 trace_id.reset(prev)
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
109 return r
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
110 return wrapper
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
111 return decorator
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
112
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
113
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
114 def store_trace_reason(location=None):
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
115 """Decorator finds and stores a reason trace was started in contextvar.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
116
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
117 Record the url for a regular web triggered request.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
118 Record the message id for an email triggered request.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
119 Record a roundup-admin command/action for roundup-admin request.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
120
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
121 Because the reason can be stored in different locations
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
122 depending on where this is called, it is called with a
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
123 location hint to activate the right extraction method.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
124
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
125 If the reason has already been stored (and it's not "missing",
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
126 it tries to extract it again and verifies it's the same as the
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
127 stored reason. If it's not the same it logs an error. This
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
128 safety check may be removed in a future version of Roundup.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
129 """
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
130 def decorator(func):
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
131 @functools.wraps(func)
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
132 def wrapper(*args, **kwargs):
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
133 nonlocal location
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
134
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
135 reason = None
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
136 prev_trace_reason = None
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
137 trace_reason = ctx_vars['trace_reason']
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
138 stored_reason = trace_reason.get()
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
139
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
140 # Fast return path. Not enabled to make sure SANITY
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
141 # CHECK below runs. If the CHECK fails, we have a a
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
142 # bad internal issue: contextvars shared between
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
143 # threads, roundup modifying reason within a request, ...
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
144 #
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
145 # if stored_reason is not _SENTINEL_REASON:
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
146 # return func(*args, **kwargs)
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
147
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
148 # use location to determine how to extract the reason
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
149 if location == "wsgi" and 'REQUEST_URI' in args[1]:
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
150 reason = args[1]['REQUEST_URI']
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
151 elif location == "client" and 'REQUEST_URI' in args[3]:
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
152 reason = args[3]['REQUEST_URI']
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
153 elif location == "mailgw":
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
154 reason = args[1].get_header('message-id', "no_message_id")
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
155 elif location == "admin":
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
156 reason = "roundup-admin(%s): %s" % (os.getlogin(), args[1][:2])
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
157 elif location.startswith("file://"):
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
158 reason = location
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
159 elif location == "client_main" and 'REQUEST_URI' in args[0].env:
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
160 reason = args[0].env['REQUEST_URI']
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
161 elif location == "xmlrpc-server":
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
162 reason = args[0].path
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
163
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
164 if reason is None:
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
165 pass
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
166 elif stored_reason is _SENTINEL_REASON:
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
167 # no value stored and reason is not none, update
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
168 prev_trace_reason = trace_reason.set(reason)
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
169 elif reason != stored_reason: # SANITY CHECK
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
170 # Throw an error we have mismatched REASON's which
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
171 # should never happen.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
172 logger.error("Mismatched REASON's: stored: %s, new: %s at %s",
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
173 stored_reason, reason, location)
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
174
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
175 try:
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
176 r = func(*args, **kwargs)
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
177 finally:
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
178 # reset context var in case thread is reused for
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
179 # another request.
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
180 if prev_trace_reason:
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
181 trace_reason.reset(prev_trace_reason)
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
182 return r
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
183 return wrapper
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
184 return decorator
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
185
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
186
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
187 def get_context_info():
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
188 """Return list of context var tuples [(var_name, var_value), ...]"""
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
189
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
190 return [(name, ctx.get()) for name, ctx in ctx_vars.items()]
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
191
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
192
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
193 #Is returning a dict for this info more pythonic?
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
194 def get_context_dict():
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
195 """Return dict of context var tuples ["var_name": "var_value", ...}"""
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
196 return {name: ctx.get() for name, ctx in ctx_vars.items()}
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
197
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
198 # Dummy no=op implementation of this module:
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
199 #
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
200 #def noop_decorator(*args, **kwargs):
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
201 # def decorator(func):
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
202 # return func
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
203 # return decorator
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
204 #
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
205 #def get_context_info():
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
206 # return [ ("trace_id", "noop_trace_id"),
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
207 # ("trace_reason", "noop_trace_reason") ]
14c7c07b32d8 feature: add thread local trace_id and trace_reason to logging.
John Rouillard <rouilj@ieee.org>
parents:
diff changeset
208 #gen_trace_id = store_trace_reason = noop_decorator
Roundup Issue Tracker: http://roundup-tracker.org/