Skip to content

[3.6] bpo-32304: Fix distutils upload for tar files ending with b'\r' (GH-5264) #5330

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merwok merged 2 commits into from
Jan 27, 2018
Merged

[3.6] bpo-32304: Fix distutils upload for tar files ending with b'\r' (GH-5264) #5330

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
fd0e963
[3.6] bpo-32304: Fix distutils upload for sdists ending with \x0d (GH…
bbayles Jan 26, 2018
d085028
bpo-32304: Port whatsnew
bbayles Jan 27, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions Doc/whatsnew/3.6.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1016,6 +1016,11 @@ attribute defaults to ``['gztar']``. Although not anticipated,
any code relying on the presence of ``default_format`` may
need to be adapted. See :issue:`27819` for more details.

The ``upload`` command now longer tries to change CR end-of-line characters
to CRLF. This fixes a corruption issue with sdists that ended with a byte
equivalent to CR.
(Contributed by Bo Bayles in :issue:`32304`.)


email
-----
Expand Down
2 changes: 0 additions & 2 deletions Lib/distutils/command/upload.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,8 +159,6 @@ def upload_file(self, command, pyversion, filename):
body.write(title.encode('utf-8'))
body.write(b"\r\n\r\n")
body.write(value)
if value and value[-1:] == b'\r':
body.write(b'\n') # write an extra newline (lurve Macs)
body.write(end_boundary)
body = body.getvalue()

Expand Down
26 changes: 26 additions & 0 deletions Lib/distutils/tests/test_upload.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,32 @@ def test_upload(self):
results = self.get_logs(INFO)
self.assertEqual(results[-1], 75 * '-' + '\nxyzzy\n' + 75 * '-')

# bpo-32304: archives whose last byte was b'\r' were corrupted due to
# normalization intended for Mac OS 9.
def test_upload_correct_cr(self):
# content that ends with \r should not be modified.
tmp = self.mkdtemp()
path = os.path.join(tmp, 'xxx')
self.write_file(path, content='yy\r')
command, pyversion, filename = 'xxx', '2.6', path
dist_files = [(command, pyversion, filename)]
self.write_file(self.rc, PYPIRC_LONG_PASSWORD)

# other fields that ended with \r used to be modified, now are
# preserved.
pkg_dir, dist = self.create_dist(
dist_files=dist_files,
description='long description\r'
)
cmd = upload(dist)
cmd.show_response = 1
cmd.ensure_finalized()
cmd.run()

headers = dict(self.last_open.req.headers)
self.assertEqual(headers['Content-length'], '2172')
self.assertIn(b'long description\r', self.last_open.req.data)

def test_upload_fails(self):
self.next_msg = "Not Found"
self.next_code = 404
Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Library/2018-01-21-16-33-53.bpo-32304.TItrNv.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
distutils' upload command no longer corrupts tar files ending with a CR byte,
and no longer tries to convert CR to CRLF in any of the upload text fields.