Skip to content

bpo-30661: Improve doc for tarfile pax change and effect on shutil #12635

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ncoghlan merged 1 commit into from
Apr 7, 2019
Merged

bpo-30661: Improve doc for tarfile pax change and effect on shutil #12635

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion Doc/library/shutil.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -559,6 +559,10 @@ provided. They rely on the :mod:`zipfile` and :mod:`tarfile` modules.

The *verbose* argument is unused and deprecated.

.. versionchanged:: 3.8
The modern pax (POSIX.1-2001) format is now used instead of
the legacy GNU format for archives created with ``format="tar"``.


.. function:: get_archive_formats()

Expand All @@ -568,7 +572,7 @@ provided. They rely on the :mod:`zipfile` and :mod:`tarfile` modules.
By default :mod:`shutil` provides these formats:

- *zip*: ZIP file (if the :mod:`zlib` module is available).
- *tar*: uncompressed tar file.
- *tar*: Uncompressed tar file. Uses POSIX.1-2001 pax format for new archives.
- *gztar*: gzip'ed tar-file (if the :mod:`zlib` module is available).
- *bztar*: bzip2'ed tar-file (if the :mod:`bz2` module is available).
- *xztar*: xz'ed tar-file (if the :mod:`lzma` module is available).
Expand Down
23 changes: 12 additions & 11 deletions Doc/library/tarfile.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -231,9 +231,9 @@ details.

The default format for creating archives. This is currently :const:`PAX_FORMAT`.

.. versionchanged:: 3.8
The default format for new archives was changed to
:const:`PAX_FORMAT` from :const:`GNU_FORMAT`.
.. versionchanged:: 3.8
The default format for new archives was changed to
:const:`PAX_FORMAT` from :const:`GNU_FORMAT`.


.. seealso::
Expand Down Expand Up @@ -813,8 +813,8 @@ Supported tar formats
There are three tar formats that can be created with the :mod:`tarfile` module:

* The POSIX.1-1988 ustar format (:const:`USTAR_FORMAT`). It supports filenames
up to a length of at best 256 characters and linknames up to 100 characters. The
maximum file size is 8 GiB. This is an old and limited but widely
up to a length of at best 256 characters and linknames up to 100 characters.
The maximum file size is 8 GiB. This is an old and limited but widely
supported format.

* The GNU tar format (:const:`GNU_FORMAT`). It supports long filenames and
Expand All @@ -826,14 +826,15 @@ There are three tar formats that can be created with the :mod:`tarfile` module:
format with virtually no limits. It supports long filenames and linknames, large
files and stores pathnames in a portable way. Modern tar implementations,
including GNU tar, bsdtar/libarchive and star, fully support extended *pax*
features; some older or unmaintained libraries may not, but should treat
features; some old or unmaintained libraries may not, but should treat
*pax* archives as if they were in the universally-supported *ustar* format.
It is the current default format for new archives.

The *pax* format is an extension to the existing *ustar* format. It uses extra
headers for information that cannot be stored otherwise. There are two flavours
of pax headers: Extended headers only affect the subsequent file header, global
headers are valid for the complete archive and affect all following files. All
the data in a pax header is encoded in *UTF-8* for portability reasons.
It extends the existing *ustar* format with extra headers for information
that cannot be stored otherwise. There are two flavours of pax headers:
Extended headers only affect the subsequent file header, global
headers are valid for the complete archive and affect all following files.
All the data in a pax header is encoded in *UTF-8* for portability reasons.

There are some more variants of the tar format which can be read, but not
created:
Expand Down
5 changes: 5 additions & 0 deletions Doc/whatsnew/3.8.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,11 @@ shutil
:func:`shutil.copytree` now accepts a new ``dirs_exist_ok`` keyword argument.
(Contributed by Josh Bronson in :issue:`20849`.)

:func:`shutil.make_archive` now defaults to the modern pax (POSIX.1-2001)
format for new archives to improve portability and standards conformance,
inherited from the corresponding change to the :mod:`tarfile` module.
(Contributed by C.A.M. Gerlach in :issue:`30661`.)


ssl
---
Expand Down