Skip to content

bpo-37111: Add 'encoding' and 'errors' parameters to logging.basicCon… #14008

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vsajip merged 2 commits into from
Jun 17, 2019
Merged

bpo-37111: Add 'encoding' and 'errors' parameters to logging.basicCon… #14008

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
76c18e2
bpo-37111: Add 'encoding' and 'errors' parameters to logging.basicCon…
vsajip Jun 12, 2019
e2dc53b
Changed errors default to 'backslashreplace'.
vsajip Jun 15, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion Doc/howto/logging.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,10 +128,18 @@ look at that next. Be sure to try the following in a newly-started Python
interpreter, and don't just continue from the session described above::

import logging
logging.basicConfig(filename='example.log',level=logging.DEBUG)
logging.basicConfig(filename='example.log', encoding='utf-8', level=logging.DEBUG)
logging.debug('This message should go to the log file')
logging.info('So should this')
logging.warning('And this, too')
logging.error('And non-ASCII stuff, too, like Øresund and Malmö')

.. versionchanged:: 3.9
The *encoding* argument was added. In earlier Python versions, or if not
specified, the encoding used is the default value used by :func:`open`. While
not shown in the above example, an *errors* argument can also now be passed,
which determines how encoding errors are handled. For available values and
the default, see the documentation for :func:`open`.
Comment thread
zooba marked this conversation as resolved.

And now if we open the file and look at what we have, we should find the log
messages:
Expand All @@ -141,6 +149,7 @@ messages:
DEBUG:root:This message should go to the log file
INFO:root:So should this
WARNING:root:And this, too
ERROR:root:And non-ASCII stuff, too, like Øresund and Malmö
Comment thread
vsajip marked this conversation as resolved.

This example also shows how you can set the logging level which acts as the
threshold for tracking. In this case, because we set the threshold to
Expand Down
34 changes: 26 additions & 8 deletions Doc/library/logging.handlers.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,18 +89,22 @@ sends logging output to a disk file. It inherits the output functionality from
:class:`StreamHandler`.


.. class:: FileHandler(filename, mode='a', encoding=None, delay=False)
.. class:: FileHandler(filename, mode='a', encoding=None, delay=False, errors=None)

@zooba zooba Jun 12, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps make it keyword only (add delay=False, *, errors=None)?


Returns a new instance of the :class:`FileHandler` class. The specified file is
opened and used as the stream for logging. If *mode* is not specified,
:const:`'a'` is used. If *encoding* is not ``None``, it is used to open the file
with that encoding. If *delay* is true, then file opening is deferred until the
first call to :meth:`emit`. By default, the file grows indefinitely.
first call to :meth:`emit`. By default, the file grows indefinitely. If
*errors* is specified, it's used to determine how encoding errors are handled.

.. versionchanged:: 3.6
As well as string values, :class:`~pathlib.Path` objects are also accepted
for the *filename* argument.

.. versionchanged:: 3.9
The *errors* parameter was added.

.. method:: close()

Closes the file.
Expand Down Expand Up @@ -168,18 +172,22 @@ exclusive locks - and so there is no need for such a handler. Furthermore,
for this value.


.. class:: WatchedFileHandler(filename, mode='a', encoding=None, delay=False)
.. class:: WatchedFileHandler(filename, mode='a', encoding=None, delay=False, errors=None)

Returns a new instance of the :class:`WatchedFileHandler` class. The specified
file is opened and used as the stream for logging. If *mode* is not specified,
:const:`'a'` is used. If *encoding* is not ``None``, it is used to open the file
with that encoding. If *delay* is true, then file opening is deferred until the
first call to :meth:`emit`. By default, the file grows indefinitely.
first call to :meth:`emit`. By default, the file grows indefinitely. If
*errors* is provided, it determines how encoding errors are handled.

.. versionchanged:: 3.6
As well as string values, :class:`~pathlib.Path` objects are also accepted
for the *filename* argument.

.. versionchanged:: 3.9
The *errors* parameter was added.

.. method:: reopenIfNeeded()

Checks to see if the file has changed. If it has, the existing stream is
Expand All @@ -205,7 +213,7 @@ module, is the base class for the rotating file handlers,
not need to instantiate this class, but it has attributes and methods you may
need to override.

.. class:: BaseRotatingHandler(filename, mode, encoding=None, delay=False)
.. class:: BaseRotatingHandler(filename, mode, encoding=None, delay=False, errors=None)

The parameters are as for :class:`FileHandler`. The attributes are:

Expand Down Expand Up @@ -284,13 +292,14 @@ The :class:`RotatingFileHandler` class, located in the :mod:`logging.handlers`
module, supports rotation of disk log files.


.. class:: RotatingFileHandler(filename, mode='a', maxBytes=0, backupCount=0, encoding=None, delay=False)
.. class:: RotatingFileHandler(filename, mode='a', maxBytes=0, backupCount=0, encoding=None, delay=False, errors=None)

Returns a new instance of the :class:`RotatingFileHandler` class. The specified
file is opened and used as the stream for logging. If *mode* is not specified,
``'a'`` is used. If *encoding* is not ``None``, it is used to open the file
with that encoding. If *delay* is true, then file opening is deferred until the
first call to :meth:`emit`. By default, the file grows indefinitely.
first call to :meth:`emit`. By default, the file grows indefinitely. If
*errors* is provided, it determines how encoding errors are handled.

You can use the *maxBytes* and *backupCount* values to allow the file to
:dfn:`rollover` at a predetermined size. When the size is about to be exceeded,
Expand All @@ -311,6 +320,9 @@ module, supports rotation of disk log files.
As well as string values, :class:`~pathlib.Path` objects are also accepted
for the *filename* argument.

.. versionchanged:: 3.9
The *errors* parameter was added.

.. method:: doRollover()

Does a rollover, as described above.
Expand All @@ -331,7 +343,7 @@ The :class:`TimedRotatingFileHandler` class, located in the
timed intervals.


.. class:: TimedRotatingFileHandler(filename, when='h', interval=1, backupCount=0, encoding=None, delay=False, utc=False, atTime=None)
.. class:: TimedRotatingFileHandler(filename, when='h', interval=1, backupCount=0, encoding=None, delay=False, utc=False, atTime=None, errors=None)

Returns a new instance of the :class:`TimedRotatingFileHandler` class. The
specified file is opened and used as the stream for logging. On rotating it also
Expand Down Expand Up @@ -391,6 +403,9 @@ timed intervals.
rollover, and subsequent rollovers would be calculated via the normal
interval calculation.

If *errors* is specified, it's used to determine how encoding errors are
handled.

.. note:: Calculation of the initial rollover time is done when the handler
is initialised. Calculation of subsequent rollover times is done only
when rollover occurs, and rollover occurs only when emitting output. If
Expand All @@ -411,6 +426,9 @@ timed intervals.
As well as string values, :class:`~pathlib.Path` objects are also accepted
for the *filename* argument.

.. versionchanged:: 3.9
The *errors* parameter was added.

.. method:: doRollover()

Does a rollover, as described above.
Expand Down
18 changes: 18 additions & 0 deletions Doc/library/logging.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1196,6 +1196,21 @@ functions.
| | carrying out the configuration as specified |
| | by the other arguments. |
+--------------+---------------------------------------------+
| *encoding* | If this keyword argument is specified along |
| | with *filename*, its value is used when the |
| | FileHandler is created, and thus used when |
| | opening the output file. |
+--------------+---------------------------------------------+
| *errors* | If this keyword argument is specified along |
| | with *filename*, its value is used when the |
| | FileHandler is created, and thus used when |
| | opening the output file. If not specified, |
| | the value 'backslashreplace' is used. Note |
| | that if ``None`` is specified, it will be |
| | passed as such to func:`open`, which means |
| | that it will be treated the same as passing |
| | 'errors'. |
+--------------+---------------------------------------------+

.. versionchanged:: 3.2
The *style* argument was added.
Expand All @@ -1209,6 +1224,9 @@ functions.
.. versionchanged:: 3.8
The *force* argument was added.

.. versionchanged:: 3.9
The *encoding* and *errors* arguments were added.

.. function:: shutdown()

Informs the logging system to perform an orderly shutdown by flushing and
Expand Down
2 changes: 2 additions & 0 deletions Doc/tools/susp-ignored.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ howto/ipaddress,,::,IPv6Address('2001:db8::ffff:ffff')
howto/ipaddress,,:ffff,IPv6Address('2001:db8::ffff:ffff')
howto/logging,,:And,"WARNING:And this, too"
howto/logging,,:And,"WARNING:root:And this, too"
howto/logging,,:And,"ERROR:root:And non-ASCII stuff, too, like "
howto/logging,,:Doing,INFO:root:Doing something
howto/logging,,:Finished,INFO:root:Finished
howto/logging,,:logger,severity:logger name:message
Expand All @@ -90,6 +91,7 @@ howto/logging,,:root,DEBUG:root:This message should go to the log file
howto/logging,,:root,INFO:root:Doing something
howto/logging,,:root,INFO:root:Finished
howto/logging,,:root,INFO:root:So should this
howto/logging,,:root,"ERROR:root:And non-ASCII stuff, too, like "
howto/logging,,:root,INFO:root:Started
howto/logging,,:root,"WARNING:root:And this, too"
howto/logging,,:root,WARNING:root:Look before you leap!
Expand Down
34 changes: 26 additions & 8 deletions Lib/logging/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2001-2017 by Vinay Sajip. All Rights Reserved.
# Copyright 2001-2019 by Vinay Sajip. All Rights Reserved.
#
# Permission to use, copy, modify, and distribute this software and its
# documentation for any purpose and without fee is hereby granted,
Expand All @@ -18,7 +18,7 @@
Logging package for Python. Based on PEP 282 and comments thereto in
comp.lang.python.

Copyright (C) 2001-2017 Vinay Sajip. All Rights Reserved.
Copyright (C) 2001-2019 Vinay Sajip. All Rights Reserved.

To use, simply 'import logging' and log away!
"""
Expand Down Expand Up @@ -1122,7 +1122,7 @@ class FileHandler(StreamHandler):
"""
A handler class which writes formatted logging records to disk files.
"""
def __init__(self, filename, mode='a', encoding=None, delay=False):
def __init__(self, filename, mode='a', encoding=None, delay=False, errors=None):
"""
Open the specified file and use it as the stream for logging.
"""
Expand All @@ -1133,6 +1133,7 @@ def __init__(self, filename, mode='a', encoding=None, delay=False):
self.baseFilename = os.path.abspath(filename)
self.mode = mode
self.encoding = encoding
self.errors = errors
self.delay = delay
if delay:
#We don't open the stream, but we still need to call the
Expand Down Expand Up @@ -1169,7 +1170,8 @@ def _open(self):
Open the current base file with the (original) mode and encoding.
Return the resulting stream.
"""
return open(self.baseFilename, self.mode, encoding=self.encoding)
return open(self.baseFilename, self.mode, encoding=self.encoding,
errors=self.errors)

def emit(self, record):
"""
Expand Down Expand Up @@ -1928,15 +1930,20 @@ def basicConfig(**kwargs):
attached to the root logger are removed and closed, before
carrying out the configuration as specified by the other
arguments.
encoding If specified together with a filename, this encoding is passed to
the created FileHandler, causing it to be used when the file is
opened.
errors If specified together with a filename, this value is passed to the
created FileHandler, causing it to be used when the file is
opened in text mode. If not specified, the default value is
`backslashreplace`.

Note that you could specify a stream created using open(filename, mode)
rather than passing the filename and mode in. However, it should be
remembered that StreamHandler does not close its stream (since it may be
using sys.stdout or sys.stderr), whereas FileHandler closes its stream
when the handler is closed.

.. versionchanged:: 3.8
Added the ``force`` parameter.

.. versionchanged:: 3.2
Added the ``style`` parameter.

Expand All @@ -1946,12 +1953,20 @@ def basicConfig(**kwargs):
``filename``/``filemode``, or ``filename``/``filemode`` specified
together with ``stream``, or ``handlers`` specified together with
``stream``.

.. versionchanged:: 3.8
Added the ``force`` parameter.

.. versionchanged:: 3.9
Added the ``encoding`` and ``errors`` parameters.
"""
# Add thread safety in case someone mistakenly calls
# basicConfig() from multiple threads
_acquireLock()
try:
force = kwargs.pop('force', False)
encoding = kwargs.pop('encoding', None)
errors = kwargs.pop('errors', 'backslashreplace')
if force:
for h in root.handlers[:]:
root.removeHandler(h)
Expand All @@ -1970,7 +1985,10 @@ def basicConfig(**kwargs):
filename = kwargs.pop("filename", None)
mode = kwargs.pop("filemode", 'a')
if filename:
h = FileHandler(filename, mode)
if 'b'in mode:
errors = None
h = FileHandler(filename, mode,
encoding=encoding, errors=errors)
else:
stream = kwargs.pop("stream", None)
h = StreamHandler(stream)
Expand Down
27 changes: 19 additions & 8 deletions Lib/logging/handlers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,13 +48,16 @@ class BaseRotatingHandler(logging.FileHandler):
Not meant to be instantiated directly. Instead, use RotatingFileHandler
or TimedRotatingFileHandler.
"""
def __init__(self, filename, mode, encoding=None, delay=False):
def __init__(self, filename, mode, encoding=None, delay=False, errors=None):
"""
Use the specified filename for streamed logging
"""
logging.FileHandler.__init__(self, filename, mode, encoding, delay)
logging.FileHandler.__init__(self, filename, mode=mode,
encoding=encoding, delay=delay,
errors=errors)
self.mode = mode
self.encoding = encoding
self.errors = errors
self.namer = None
self.rotator = None

Expand Down Expand Up @@ -117,7 +120,8 @@ class RotatingFileHandler(BaseRotatingHandler):
Handler for logging to a set of files, which switches from one file
to the next when the current file reaches a certain size.
"""
def __init__(self, filename, mode='a', maxBytes=0, backupCount=0, encoding=None, delay=False):
def __init__(self, filename, mode='a', maxBytes=0, backupCount=0,
encoding=None, delay=False, errors=None):
"""
Open the specified file and use it as the stream for logging.

Expand Down Expand Up @@ -145,7 +149,8 @@ def __init__(self, filename, mode='a', maxBytes=0, backupCount=0, encoding=None,
# on each run.
if maxBytes > 0:
mode = 'a'
BaseRotatingHandler.__init__(self, filename, mode, encoding, delay)
BaseRotatingHandler.__init__(self, filename, mode, encoding=encoding,
delay=delay, errors=errors)
self.maxBytes = maxBytes
self.backupCount = backupCount

Expand Down Expand Up @@ -196,8 +201,11 @@ class TimedRotatingFileHandler(BaseRotatingHandler):
If backupCount is > 0, when rollover is done, no more than backupCount
files are kept - the oldest ones are deleted.
"""
def __init__(self, filename, when='h', interval=1, backupCount=0, encoding=None, delay=False, utc=False, atTime=None):
BaseRotatingHandler.__init__(self, filename, 'a', encoding, delay)
def __init__(self, filename, when='h', interval=1, backupCount=0,
encoding=None, delay=False, utc=False, atTime=None,
errors=None):
BaseRotatingHandler.__init__(self, filename, 'a', encoding=encoding,
delay=delay, errors=errors)
self.when = when.upper()
self.backupCount = backupCount
self.utc = utc
Expand Down Expand Up @@ -431,8 +439,11 @@ class WatchedFileHandler(logging.FileHandler):
This handler is based on a suggestion and patch by Chad J.
Schroeder.
"""
def __init__(self, filename, mode='a', encoding=None, delay=False):
logging.FileHandler.__init__(self, filename, mode, encoding, delay)
def __init__(self, filename, mode='a', encoding=None, delay=False,
errors=None):
logging.FileHandler.__init__(self, filename, mode=mode,
encoding=encoding, delay=delay,
errors=errors)
self.dev, self.ino = -1, -1
self._statstream()

Expand Down
Loading