Skip to content

Commit 6090187

Browse files
ezio-melottiezio-melotti
committed
#10535: Enable silenced warnings in unittest by default
1 parent 00f2f97 commit 6090187

8 files changed

Lines changed: 228 additions & 22 deletions

File tree

‎Doc/library/unittest.rst‎

Lines changed: 20 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1845,12 +1845,21 @@ Loading and running tests
18451845
instead of repeatedly creating new instances.
18461846

18471847

1848-
.. class:: TextTestRunner(stream=sys.stderr, descriptions=True, verbosity=1, runnerclass=None)
1848+
.. class:: TextTestRunner(stream=sys.stderr, descriptions=True, verbosity=1, runnerclass=None, warnings=None)
18491849

18501850
A basic test runner implementation which prints results on standard error. It
18511851
has a few configurable parameters, but is essentially very simple. Graphical
18521852
applications which run test suites should provide alternate implementations.
18531853

1854+
By default this runner shows :exc:`DeprecationWarning`,
1855+
:exc:`PendingDeprecationWarning`, and :exc:`ImportWarning` even if they are
1856+
:ref:`ignored by default <warning-ignored>`. Deprecation warnings caused by
1857+
:ref:`deprecated unittest methods <deprecated-aliases>` are also
1858+
special-cased and, when the warning filters are ``'default'`` or ``'always'``,
1859+
they will appear only once per-module, in order to avoid too many warning
1860+
messages. This behavior can be overridden using the :option`-Wd` or
1861+
:option:`-Wa` options and leaving *warnings* to ``None``.
1862+
18541863
.. method:: _makeResult()
18551864

18561865
This method returns the instance of ``TestResult`` used by :meth:`run`.
@@ -1864,7 +1873,9 @@ Loading and running tests
18641873

18651874
stream, descriptions, verbosity
18661875

1867-
.. function:: main(module='__main__', defaultTest=None, argv=None, testRunner=None, testLoader=unittest.loader.defaultTestLoader, exit=True, verbosity=1, failfast=None, catchbreak=None, buffer=None)
1876+
.. versionchanged:: 3.2 Added the ``warnings`` argument
1877+
1878+
.. function:: main(module='__main__', defaultTest=None, argv=None, testRunner=None, testLoader=unittest.loader.defaultTestLoader, exit=True, verbosity=1, failfast=None, catchbreak=None, buffer=None, warnings=None)
18681879

18691880
A command-line program that runs a set of tests; this is primarily for making
18701881
test modules conveniently executable. The simplest use for this function is to
@@ -1893,12 +1904,17 @@ Loading and running tests
18931904
The ``failfast``, ``catchbreak`` and ``buffer`` parameters have the same
18941905
effect as the same-name `command-line options`_.
18951906

1907+
The *warning* argument specifies the :ref:`warning filter <warning-filter>`
1908+
that should be used while running the tests. If it's not specified, it will
1909+
remain ``None`` if a :option:`-W` option is passed to :program:`python`,
1910+
otherwise it will be set to ``'default'``.
1911+
18961912
Calling ``main`` actually returns an instance of the ``TestProgram`` class.
18971913
This stores the result of the tests run as the ``result`` attribute.
18981914

18991915
.. versionchanged:: 3.2
1900-
The ``exit``, ``verbosity``, ``failfast``, ``catchbreak`` and ``buffer``
1901-
parameters were added.
1916+
The ``exit``, ``verbosity``, ``failfast``, ``catchbreak``, ``buffer``,
1917+
and ``warnings`` parameters were added.
19021918

19031919

19041920
load_tests Protocol

‎Doc/library/warnings.rst‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -249,6 +249,8 @@ continues to increase after each operation, or else delete the previous
249249
entries from the warnings list before each new operation).
250250

251251

252+
.. _warning-ignored:
253+
252254
Updating Code For New Versions of Python
253255
----------------------------------------
254256

@@ -279,6 +281,9 @@ code that were not there in an older interpreter, e.g.
279281
developer want to be notified that your code is using a deprecated module, to a
280282
user this information is essentially noise and provides no benefit to them.
281283

284+
The :mod:`unittest` module has been also updated to use the ``'default'``
285+
filter while running tests.
286+
282287

283288
.. _warning-functions:
284289

‎Lib/unittest/main.py‎

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -67,12 +67,12 @@ class TestProgram(object):
6767
USAGE = USAGE_FROM_MODULE
6868

6969
# defaults for testing
70-
failfast = catchbreak = buffer = progName = None
70+
failfast = catchbreak = buffer = progName = warnings = None
7171

7272
def __init__(self, module='__main__', defaultTest=None, argv=None,
7373
testRunner=None, testLoader=loader.defaultTestLoader,
7474
exit=True, verbosity=1, failfast=None, catchbreak=None,
75-
buffer=None):
75+
buffer=None, warnings=None):
7676
if isinstance(module, str):
7777
self.module = __import__(module)
7878
for part in module.split('.')[1:]:
@@ -87,6 +87,18 @@ def __init__(self, module='__main__', defaultTest=None, argv=None,
8787
self.catchbreak = catchbreak
8888
self.verbosity = verbosity
8989
self.buffer = buffer
90+
if warnings is None and not sys.warnoptions:
91+
# even if DreprecationWarnings are ignored by default
92+
# print them anyway unless other warnings settings are
93+
# specified by the warnings arg or the -W python flag
94+
self.warnings = 'default'
95+
else:
96+
# here self.warnings is set either to the value passed
97+
# to the warnings args or to None.
98+
# If the user didn't pass a value self.warnings will
99+
# be None. This means that the behavior is unchanged
100+
# and depends on the values passed to -W.
101+
self.warnings = warnings
90102
self.defaultTest = defaultTest
91103
self.testRunner = testRunner
92104
self.testLoader = testLoader
@@ -220,7 +232,8 @@ def runTests(self):
220232
try:
221233
testRunner = self.testRunner(verbosity=self.verbosity,
222234
failfast=self.failfast,
223-
buffer=self.buffer)
235+
buffer=self.buffer,
236+
warnings=self.warnings)
224237
except TypeError:
225238
# didn't accept the verbosity, buffer or failfast arguments
226239
testRunner = self.testRunner()

‎Lib/unittest/runner.py‎

Lines changed: 27 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
import sys
44
import time
5+
import warnings
56

67
from . import result
78
from .signals import registerResult
@@ -125,12 +126,13 @@ class TextTestRunner(object):
125126
resultclass = TextTestResult
126127

127128
def __init__(self, stream=sys.stderr, descriptions=True, verbosity=1,
128-
failfast=False, buffer=False, resultclass=None):
129+
failfast=False, buffer=False, resultclass=None, warnings=None):
129130
self.stream = _WritelnDecorator(stream)
130131
self.descriptions = descriptions
131132
self.verbosity = verbosity
132133
self.failfast = failfast
133134
self.buffer = buffer
135+
self.warnings = warnings
134136
if resultclass is not None:
135137
self.resultclass = resultclass
136138

@@ -143,17 +145,30 @@ def run(self, test):
143145
registerResult(result)
144146
result.failfast = self.failfast
145147
result.buffer = self.buffer
146-
startTime = time.time()
147-
startTestRun = getattr(result, 'startTestRun', None)
148-
if startTestRun is not None:
149-
startTestRun()
150-
try:
151-
test(result)
152-
finally:
153-
stopTestRun = getattr(result, 'stopTestRun', None)
154-
if stopTestRun is not None:
155-
stopTestRun()
156-
stopTime = time.time()
148+
with warnings.catch_warnings():
149+
if self.warnings:
150+
# if self.warnings is set, use it to filter all the warnings
151+
warnings.simplefilter(self.warnings)
152+
# if the filter is 'default' or 'always', special-case the
153+
# warnings from the deprecated unittest methods to show them
154+
# no more than once per module, because they can be fairly
155+
# noisy. The -Wd and -Wa flags can be used to bypass this
156+
# only when self.warnings is None.
157+
if self.warnings in ['default', 'always']:
158+
warnings.filterwarnings('module',
159+
category=DeprecationWarning,
160+
message='Please use assert\w+ instead.')
161+
startTime = time.time()
162+
startTestRun = getattr(result, 'startTestRun', None)
163+
if startTestRun is not None:
164+
startTestRun()
165+
try:
166+
test(result)
167+
finally:
168+
stopTestRun = getattr(result, 'stopTestRun', None)
169+
if stopTestRun is not None:
170+
stopTestRun()
171+
stopTime = time.time()
157172
timeTaken = stopTime - startTime
158173
result.printErrors()
159174
if hasattr(result, 'separator2'):

‎Lib/unittest/test/_test_warnings.py‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# helper module for test_runner.Test_TextTestRunner.test_warnings
2+
3+
"""
4+
This module has a number of tests that raise different kinds of warnings.
5+
When the tests are run, the warnings are caught and their messages are printed
6+
to stdout. This module also accepts an arg that is then passed to
7+
unittest.main to affect the behavior of warnings.
8+
Test_TextTestRunner.test_warnings executes this script with different
9+
combinations of warnings args and -W flags and check that the output is correct.
10+
See #10535.
11+
"""
12+
13+
import io
14+
import sys
15+
import unittest
16+
import warnings
17+
18+
def warnfun():
19+
warnings.warn('rw', RuntimeWarning)
20+
21+
class TestWarnings(unittest.TestCase):
22+
# unittest warnings will be printed at most once per type (max one message
23+
# for the fail* methods, and one for the assert* methods)
24+
def test_assert(self):
25+
self.assertEquals(2+2, 4)
26+
self.assertEquals(2*2, 4)
27+
self.assertEquals(2**2, 4)
28+
29+
def test_fail(self):
30+
self.failUnless(1)
31+
self.failUnless(True)
32+
33+
def test_other_unittest(self):
34+
self.assertAlmostEqual(2+2, 4)
35+
self.assertNotAlmostEqual(4+4, 2)
36+
37+
# these warnings are normally silenced, but they are printed in unittest
38+
def test_deprecation(self):
39+
warnings.warn('dw', DeprecationWarning)
40+
warnings.warn('dw', DeprecationWarning)
41+
warnings.warn('dw', DeprecationWarning)
42+
43+
def test_import(self):
44+
warnings.warn('iw', ImportWarning)
45+
warnings.warn('iw', ImportWarning)
46+
warnings.warn('iw', ImportWarning)
47+
48+
# user warnings should always be printed
49+
def test_warning(self):
50+
warnings.warn('uw')
51+
warnings.warn('uw')
52+
warnings.warn('uw')
53+
54+
# these warnings come from the same place; they will be printed
55+
# only once by default or three times if the 'always' filter is used
56+
def test_function(self):
57+
58+
warnfun()
59+
warnfun()
60+
warnfun()
61+
62+
63+
64+
if __name__ == '__main__':
65+
with warnings.catch_warnings(record=True) as ws:
66+
# if an arg is provided pass it to unittest.main as 'warnings'
67+
if len(sys.argv) == 2:
68+
unittest.main(exit=False, warnings=sys.argv.pop())
69+
else:
70+
unittest.main(exit=False)
71+
72+
# print all the warning messages collected
73+
for w in ws:
74+
print(w.message)

‎Lib/unittest/test/test_break.py‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -209,7 +209,8 @@ def __init__(self, catchbreak):
209209

210210
self.assertEqual(FakeRunner.initArgs, [((), {'buffer': None,
211211
'verbosity': verbosity,
212-
'failfast': failfast})])
212+
'failfast': failfast,
213+
'warnings': None})])
213214
self.assertEqual(FakeRunner.runArgs, [test])
214215
self.assertEqual(p.result, result)
215216

@@ -222,7 +223,8 @@ def __init__(self, catchbreak):
222223

223224
self.assertEqual(FakeRunner.initArgs, [((), {'buffer': None,
224225
'verbosity': verbosity,
225-
'failfast': failfast})])
226+
'failfast': failfast,
227+
'warnings': None})])
226228
self.assertEqual(FakeRunner.runArgs, [test])
227229
self.assertEqual(p.result, result)
228230

‎Lib/unittest/test/test_program.py‎

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -182,19 +182,42 @@ def testBufferCatchFailfast(self):
182182
program.parseArgs([None, opt])
183183
self.assertEqual(getattr(program, attr), not_none)
184184

185+
def testWarning(self):
186+
"""Test the warnings argument"""
187+
# see #10535
188+
class FakeTP(unittest.TestProgram):
189+
def parseArgs(self, *args, **kw): pass
190+
def runTests(self, *args, **kw): pass
191+
warnoptions = sys.warnoptions
192+
try:
193+
sys.warnoptions[:] = []
194+
# no warn options, no arg -> default
195+
self.assertEqual(FakeTP().warnings, 'default')
196+
# no warn options, w/ arg -> arg value
197+
self.assertEqual(FakeTP(warnings='ignore').warnings, 'ignore')
198+
sys.warnoptions[:] = ['somevalue']
199+
# warn options, no arg -> None
200+
# warn options, w/ arg -> arg value
201+
self.assertEqual(FakeTP().warnings, None)
202+
self.assertEqual(FakeTP(warnings='ignore').warnings, 'ignore')
203+
finally:
204+
sys.warnoptions[:] = warnoptions
205+
185206
def testRunTestsRunnerClass(self):
186207
program = self.program
187208

188209
program.testRunner = FakeRunner
189210
program.verbosity = 'verbosity'
190211
program.failfast = 'failfast'
191212
program.buffer = 'buffer'
213+
program.warnings = 'warnings'
192214

193215
program.runTests()
194216

195217
self.assertEqual(FakeRunner.initArgs, {'verbosity': 'verbosity',
196218
'failfast': 'failfast',
197-
'buffer': 'buffer'})
219+
'buffer': 'buffer',
220+
'warnings': 'warnings'})
198221
self.assertEqual(FakeRunner.test, 'test')
199222
self.assertIs(program.result, RESULT)
200223

‎Lib/unittest/test/test_runner.py‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
import io
2+
import os
3+
import sys
24
import pickle
5+
import subprocess
36

47
import unittest
58

@@ -144,6 +147,7 @@ def test_init(self):
144147
self.assertFalse(runner.failfast)
145148
self.assertFalse(runner.buffer)
146149
self.assertEqual(runner.verbosity, 1)
150+
self.assertEqual(runner.warnings, None)
147151
self.assertTrue(runner.descriptions)
148152
self.assertEqual(runner.resultclass, unittest.TextTestResult)
149153

@@ -244,3 +248,57 @@ def MockResultClass(*args):
244248

245249
expectedresult = (runner.stream, DESCRIPTIONS, VERBOSITY)
246250
self.assertEqual(runner._makeResult(), expectedresult)
251+
252+
def test_warnings(self):
253+
"""
254+
Check that warnings argument of TextTestRunner correctly affects the
255+
behavior of the warnings.
256+
"""
257+
# see #10535 and the _test_warnings file for more information
258+
259+
def get_parse_out_err(p):
260+
return [b.splitlines() for b in p.communicate()]
261+
opts = dict(stdout=subprocess.PIPE, stderr=subprocess.PIPE,
262+
cwd=os.path.dirname(__file__))
263+
ae_msg = b'Please use assertEqual instead.'
264+
at_msg = b'Please use assertTrue instead.'
265+
266+
# no args -> all the warnings are printed, unittest warnings only once
267+
p = subprocess.Popen([sys.executable, '_test_warnings.py'], **opts)
268+
out, err = get_parse_out_err(p)
269+
self.assertEqual(err[-1], b'OK')
270+
# check that the total number of warnings in the output is correct
271+
self.assertEqual(len(out), 12)
272+
# check that the numbers of the different kind of warnings is correct
273+
for msg in [b'dw', b'iw', b'uw']:
274+
self.assertEqual(out.count(msg), 3)
275+
for msg in [ae_msg, at_msg, b'rw']:
276+
self.assertEqual(out.count(msg), 1)
277+
278+
args_list = (
279+
# passing 'ignore' as warnings arg -> no warnings
280+
[sys.executable, '_test_warnings.py', 'ignore'],
281+
# -W doesn't affect the result if the arg is passed
282+
[sys.executable, '-Wa', '_test_warnings.py', 'ignore'],
283+
# -W affects the result if the arg is not passed
284+
[sys.executable, '-Wi', '_test_warnings.py']
285+
)
286+
# in all these cases no warnings are printed
287+
for args in args_list:
288+
p = subprocess.Popen(args, **opts)
289+
out, err = get_parse_out_err(p)
290+
self.assertEqual(err[-1], b'OK')
291+
self.assertEqual(len(out), 0)
292+
293+
294+
# passing 'always' as warnings arg -> all the warnings printed,
295+
# unittest warnings only once
296+
p = subprocess.Popen([sys.executable, '_test_warnings.py', 'always'],
297+
**opts)
298+
out, err = get_parse_out_err(p)
299+
self.assertEqual(err[-1], b'OK')
300+
self.assertEqual(len(out), 14)
301+
for msg in [b'dw', b'iw', b'uw', b'rw']:
302+
self.assertEqual(out.count(msg), 3)
303+
for msg in [ae_msg, at_msg]:
304+
self.assertEqual(out.count(msg), 1)

0 commit comments

Comments
 (0)