Repository navigation
Closed
Labels
extension-modulesC modules in the Modules dirC modules in the Modules dirrelease-blockertopic-IOtype-securityA security issueA security issue
Description
| BPO | 24989 |
|---|---|
| Nosy | @vstinner, @larryhastings, @vadmium, @serhiy-storchaka |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2015-09-04.05:13:49.956>
created_at = <Date 2015-09-02.23:40:26.639>
labels = ['type-security', 'extension-modules', 'expert-IO', 'release-blocker']
title = 'scan_eol() Buffer Over-read'
updated_at = <Date 2015-09-08.18:34:24.218>
user = 'https://bugs.python.org/JohnLeitch'bugs.python.org fields:
activity = <Date 2015-09-08.18:34:24.218>
actor = 'Arfrever'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2015-09-04.05:13:49.956>
closer = 'larry'
components = ['Extension Modules', 'IO']
creation = <Date 2015-09-02.23:40:26.639>
creator = 'JohnLeitch'
dependencies = []
files = ['40326', '40327', '40330']
hgrepos = []
issue_num = 24989
keywords = ['patch']
message_count = 11.0
messages = ['249584', '249600', '249602', '249606', '249610', '249617', '249699', '249711', '249717', '249718', '249738']
nosy_count = 8.0
nosy_names = ['vstinner', 'larry', 'Arfrever', 'python-dev', 'martin.panter', 'serhiy.storchaka', 'JohnLeitch', 'brycedarling']
pr_nums = []
priority = 'release blocker'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'security'
url = 'https://bugs.python.org/issue24989'
versions = ['Python 3.5', 'Python 3.6']Metadata
Metadata
Assignees
Labels
extension-modulesC modules in the Modules dirC modules in the Modules dirrelease-blockertopic-IOtype-securityA security issueA security issue
Activity
JohnLeitch commentedon Sep 2, 2015
Python 3.5 suffers from a vulnerability caused by the behavior of the scan_eol() function. When called, the function gets a line from the buffer of a BytesIO object by searching for a newline character starting at the position in the buffer.
However, if the position is set to a value that is larger than the buffer, this logic will result in a call to memchr that reads off the end of the buffer:
In some applications, it may be possible to exploit this behavior to disclose the contents of adjacent memory. The buffer over-read can be observed by running the following script:
Which, depending on the arrangement of memory, may produce an exception such as this:
0:000> g
(698.188): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=fff8a14c ebx=0a0a0a0a ecx=00000000 edx=05bb1000 esi=061211b0 edi=89090909
eip=61c6caf2 esp=010af8dc ebp=010af914 iopl=0 nv up ei ng nz ac po nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010292
python35!memchr+0x62:
61c6caf2 8b0a mov ecx,dword ptr [edx] ds:002b:05bb1000=????????
0:000> k1
ChildEBP RetAddr
010af8e 61b640f1 python35!memchr+0x62 [f:\dd\vctools\crt_bld\SELF_X86\crt\src\INTEL\memchr.asm @ 125]
To fix this issue, it is recommended that scan_eol() be updated to check that the position is not greater than or equal to the size of the buffer. A proposed patch is attached.
vadmium commentedon Sep 3, 2015
Simpler test case, which might find a place somewhere like /Lib/test/test_memoryio.py:
The patch looks like the right approach. Just watch out for the indentation (tabs vs spaces).
However I don’t see why we need the GET_BYTES_SIZE() macro or the (size_t) casting. Would it be okay comparing PyBytes_GET_SIZE() directly to self->pos? They are both meant to be Py_ssize_t.
The bug looks like it was introduced by Serhiy’s changes in bpo-15381.
JohnLeitch commentedon Sep 3, 2015
We based our fix on the check in write_bytes:
I see now that our casting was extraneous. As for the macro, it was suspected that similar issues may be present and we wanted to write reusable code, but this also seems unnecessary now that it's known the cast is unneeded.
Early tomorrow I'll take some time to create a revised patch.
serhiy-storchaka commentedon Sep 3, 2015
It should be self->string_size, not PyBytes_GET_SIZE(self->buf).
Thank you for your report and your patch John. Here is revised patch with tests based on Martin's test.
Larry, perhaps this bug is grave enough to be fixed in RC3.
vstinner commentedon Sep 3, 2015
scan_eol_Buffer_Over-read_2.patch looks good to me.
Since it looks like a regression in Python 3.5, yes, it's a severe bug. Please send a pull request to Larry for it.
larryhastings commentedon Sep 3, 2015
Yes, please create a pull request for this patch. Thanks!
And just to confirm: I just applied patch 2 to CPython, then undid the change to bytesio.c. The new test fails, and sometimes Python will segmentation fault. If I then apply the patch to bytesio.c it passes. Nice work!
python-dev commentedon Sep 3, 2015
New changeset a5858c30db7c by Serhiy Storchaka in branch '3.5':
Issue bpo-24989: Fixed buffer overread in BytesIO.readline() if a position is
https://hg.python.org/cpython/rev/a5858c30db7c
New changeset 215800fb955d by Serhiy Storchaka in branch 'default':
Issue bpo-24989: Fixed buffer overread in BytesIO.readline() if a position is
https://hg.python.org/cpython/rev/215800fb955d
serhiy-storchaka commentedon Sep 4, 2015
https://bitbucket.org/larry/cpython350/pull-requests/13/issue-24989/diff
larryhastings commentedon Sep 4, 2015
Pull request accepted. Please forward-merge. Thanks!
python-dev commentedon Sep 4, 2015
New changeset 2b6ce7e9595c by Serhiy Storchaka in branch '3.5':
Issue bpo-24989: Fixed buffer overread in BytesIO.readline() if a position is
https://hg.python.org/cpython/rev/2b6ce7e9595c
python-dev commentedon Sep 4, 2015
New changeset 07e04c34bab5 by Larry Hastings in branch '3.5':
Merged in storchaka/cpython350 (pull request #13)
https://hg.python.org/cpython/rev/07e04c34bab5