Skip to content

[3.6] bpo-32228: Reset raw_pos after unwinding the raw stream (GH-4858) #5389

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pitrou merged 2 commits into from
Jan 28, 2018
Merged

[3.6] bpo-32228: Reset raw_pos after unwinding the raw stream (GH-4858) #5389

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
72523e3
[3.6] bpo-32228: Reset raw_pos after unwinding the raw stream (GH-4858)
nitishch Jan 28, 2018
bd43d6f
[3.6] bpo-32228: Reset raw_pos after unwinding the raw stream (GH-4858)
pitrou Jan 28, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions Lib/test/test_io.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1703,6 +1703,23 @@ def test_truncate(self):
with self.open(support.TESTFN, "rb", buffering=0) as f:
self.assertEqual(f.read(), b"abc")

def test_truncate_after_write(self):
# Ensure that truncate preserves the file position after
# writes longer than the buffer size.
# Issue: https://bugs.python.org/issue32228
with self.open(support.TESTFN, "wb") as f:
# Fill with some buffer
f.write(b'\x00' * 10000)
buffer_sizes = [8192, 4096, 200]
for buffer_size in buffer_sizes:
with self.open(support.TESTFN, "r+b", buffering=buffer_size) as f:
f.write(b'\x00' * (buffer_size + 1))
# After write write_pos and write_end are set to 0
f.read(1)
# read operation makes sure that pos != raw_pos
f.truncate()
self.assertEqual(f.tell(), buffer_size + 2)

@unittest.skipUnless(threading, 'Threading required for this test.')
@support.requires_resource('cpu')
def test_threads(self):
Expand Down
1 change: 1 addition & 0 deletions Misc/NEWS.d/next/Library/2017-12-22-16-47-41.bpo-32228.waPx3q.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Ensure that ``truncate()`` preserves the file position (as reported by ``tell()``) after writes longer than the buffer size.
14 changes: 10 additions & 4 deletions Modules/_io/bufferedio.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1311,7 +1311,6 @@ _io__Buffered_seek_impl(buffered *self, PyObject *targetobj, int whence)
if (res == NULL)
goto end;
Py_CLEAR(res);
_bufferedwriter_reset_buf(self);
}

/* TODO: align on block boundary and read buffer if needed? */
Expand Down Expand Up @@ -1878,8 +1877,6 @@ _bufferedwriter_raw_write(buffered *self, char *start, Py_ssize_t len)
return n;
}

/* `restore_pos` is 1 if we need to restore the raw stream position at
the end, 0 otherwise. */
static PyObject *
_bufferedwriter_flush_unlocked(buffered *self)
{
Expand Down Expand Up @@ -1920,9 +1917,18 @@ _bufferedwriter_flush_unlocked(buffered *self)
goto error;
}

_bufferedwriter_reset_buf(self);

end:
/* This ensures that after return from this function,
VALID_WRITE_BUFFER(self) returns false.

This is a required condition because when a tell() is called
after flushing and if VALID_READ_BUFFER(self) is false, we need
VALID_WRITE_BUFFER(self) to be false to have
RAW_OFFSET(self) == 0.

Issue: https://bugs.python.org/issue32228 */
_bufferedwriter_reset_buf(self);
Py_RETURN_NONE;

error:
Expand Down