Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
c9b880a
Code and documentation for HTTP compression support in http.server
Jun 9, 2017
481f69e
Merge remote-tracking branch 'upstream2/master'
Jun 9, 2017
5a3693e
Implementation change : if original file is big, store zipped content…
Jun 9, 2017
bed3047
Resolve conflict
Jun 9, 2017
d82b7cd
resolve conflict in NEWS
Jun 10, 2017
e9deb89
For gzipped content on disk, set file pointer at position 0.
Jun 10, 2017
39da21b
Merge branch 'master' into master
PierreQuentel Jun 11, 2017
f9963f0
Merge remote-tracking branch 'upstream2/master'
Jun 12, 2017
ace7999
resolve conflict in NEWS
Jun 12, 2017
128f6a6
Fix conflicts
Jun 12, 2017
586b8c3
Add HTTP compression test for big files
Jun 12, 2017
a61109e
Merge branch 'master' into master
PierreQuentel Jun 15, 2017
6093cae
Merge branch 'master' into master
PierreQuentel Jun 17, 2017
b27144a
Support more Accept-Encoding values, eg "*", "gzip;q=0.5", "GZIP"
Jun 18, 2017
3525f56
Merge remote-tracking branch 'upstream2/master'
Jun 18, 2017
1ad8139
Merge branch 'master' of https://github.com/PierreQuentel/cpython
Jun 18, 2017
8a916fc
More documentation
Jun 18, 2017
6f193f9
Remove trailing whitespace
Jun 18, 2017
655f753
Handle the case when storing the temporary gzip file fails, eg for la…
Jun 19, 2017
2433769
Add comment.
Jun 24, 2017
cfdcf93
Remove changes to mimetypes.py
Jul 1, 2017
9d8e668
Restore tests in test_httpservers, remove test with .json extension
Jul 1, 2017
b700014
Remove whitespaces
Jul 1, 2017
6a31d60
Remove whitespaces
Jul 1, 2017
9830b89
Remove whitespaces
Jul 1, 2017
6466c93
Merge remote-tracking branch 'upstream2/master'
Jul 25, 2017
89de0fe
Disable HTTP compression by default. Add command line option --gzip t…
Jul 25, 2017
211bf66
Remove trailing whitespace
Jul 25, 2017
a7f1f47
Only apply chunk transfer for HTTP/1.1 ; change implementation of com…
Jul 27, 2017
79cec36
Simplify code for HTTP compression
Jul 28, 2017
83f6082
Add tests on presence of Transfer-Encoding header
Jul 28, 2017
329d5f8
Merge remote-tracking branch 'upstream2/master'
Jul 28, 2017
2ca373f
Add entry in NEWS.d
Jul 28, 2017
8fbc454
Merge remote-tracking branch 'upstream2/master'
Jul 28, 2017
e51cb95
Merge remote-tracking branch 'upstream2/master'
Aug 2, 2017
cfb599d
Remove variable has_gzip, set gzip to None in case of ImportError. Us…
Aug 2, 2017
92a47f7
Merge remote-tracking branch 'upstream2/master'
Aug 13, 2017
75b78a1
By default, "deflate" is supported besides "gzip" ; other compression…
Aug 13, 2017
5db66e0
Add test for user-defined compression (bzip2).
Aug 13, 2017
059a745
Remove trailing whitespace.
Aug 13, 2017
c099836
Merge remote-tracking branch 'upstream2/master'
Sep 13, 2017
396866d
Update http.server documentation
Sep 15, 2017
553c920
Merge remote-tracking branch 'upstream2/master'
Sep 15, 2017
9fdecb1
Resolve conflict
Sep 15, 2017
f0293fc
Restore Lib/test/bisect.py
Sep 15, 2017
fc2d9ca
Remove Misc/NEWS
Sep 15, 2017
baf88e4
Remove unused import (tempfile)
Oct 1, 2017
2bce4ef
Replace "as argument" by "as an argument"
Oct 1, 2017
1bdbb3a
Remove unused import (gzip)
Oct 1, 2017
27d02bd
If zlib is not available, set SimpleHTTPRequestHandler.compressions t…
Oct 1, 2017
ca68881
Replace sorted(...)[-1] by max(...)
Oct 1, 2017
d6157d4
For shutil.rmtree, instead of a try/except, set argument ignore_error…
Oct 1, 2017
a456fe6
Replace assertTrue(a in b) by assertIn(a, b) and assertFalse(a in b) …
Oct 1, 2017
50f0a85
Remove default_request_version
Oct 1, 2017
87a2cc3
Merge remote-tracking branch 'upstream2/master'
Oct 1, 2017
b9f9599
Restore Lib/test/bisect.py
Oct 1, 2017
7881e1f
Compressed data generators may send empty bytes ; adapt do_GET() for …
Oct 1, 2017
2332d91
Remove trace
Oct 1, 2017
70f4738
If Accept-Encoding is set to *, use one of the supported compressions…
Oct 1, 2017
d44c8bf
Minor changes in comments and code formatting.
Oct 1, 2017
d844ca6
Adapt test with Accept-Encoding set to "*"
Oct 1, 2017
634f572
Merge remote-tracking branch 'upstream2/master'
Oct 11, 2017
7cfa86e
Handle missing zlib or bz2 modules
Oct 11, 2017
829b992
Replace command line option "--gzip" by "--compressed"
Oct 11, 2017
383ee33
Improve documentation for attribute "compressions" ; replace command …
Oct 11, 2017
9d04caa
Merge branch 'master' into master
PierreQuentel Apr 16, 2018
71e9090
Merge branch 'master' into master
PierreQuentel Jul 30, 2018
d71b3f4
bpo-20504: bug in cgi.py when a multipart/form-data request has no co…
Nov 21, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 38 additions & 3 deletions Doc/library/http.server.rst
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,6 @@ provides three different variants:
the message to :meth:`log_message`, so it takes the same arguments
(*format* and additional values).


.. method:: log_message(format, ...)

Logs an arbitrary message to ``sys.stderr``. This is typically overridden
Expand Down Expand Up @@ -338,6 +337,29 @@ provides three different variants:

If not specified, the directory to serve is the current working directory.

.. attribute:: compressed_types

The list of content types for which HTTP compression is applied. Set by
default to the empty list, which means that by default, compression is
disabled. It can be set to a list of types, for instance :
``["text/plain", "text/html", "text/css"]``. A list of commonly
compressed types is provided as ``commonly_compressed_types`` at the
module level.

.. versionadded:: 3.7

.. attribute:: compressions

A mapping between a compression encoding (eg. "gzip") and a generator
that takes a file object as an argument, reads the file content by
chunks, compresses each chunk with the specified encoding, and yields
the compressed data as a bytes object.
By default, if the :mod:`zlib` module is available, "gzip" and "deflate"
compressions are supported. To support other algorithms,
:attr:`compressions` can be extended.

.. versionadded:: 3.7

The :class:`SimpleHTTPRequestHandler` class defines the following methods:

.. method:: do_HEAD()
Expand Down Expand Up @@ -366,6 +388,12 @@ provides three different variants:
type is guessed by calling the :meth:`guess_type` method, which in turn
uses the *extensions_map* variable, and the file contents are returned.

If the content type is in the list ``compressed_types``, and if the
user agent has sent an ``'Accept-Encoding'`` header that included
"gzip", a header ``'Content-Encoding'`` set to "gzip" is sent and the
file content is compressed using gzip. For big files, the gzipped
content is stored in a temporary file.

A ``'Content-type:'`` header with the guessed content type is output,
followed by a ``'Content-Length:'`` header with the file's size and a
``'Last-Modified:'`` header with the file's modification time.
Expand All @@ -377,8 +405,9 @@ provides three different variants:
For example usage, see the implementation of the :func:`test` function
invocation in the :mod:`http.server` module.

.. versionchanged:: 3.7
Support of the ``'If-Modified-Since'`` header.
.. versionadded:: 3.7
Support of the ``'If-Modified-Since'`` header and of HTTP
compression.

The :class:`SimpleHTTPRequestHandler` class can be used in the following
manner in order to create a very basic webserver serving files relative to
Expand Down Expand Up @@ -421,6 +450,12 @@ the following command uses a specific directory::
.. versionadded:: 3.7
``--directory`` specify alternate directory

By default, HTTP compression is not supported. Setting the option
``--compressed`` enables compression on the content types defined in
``commonly_compressed_types``.

.. versionadded:: 3.7

.. class:: CGIHTTPRequestHandler(request, client_address, server)

This class is used to serve either files or output of CGI scripts from the
Expand Down
14 changes: 14 additions & 0 deletions Doc/whatsnew/3.7.rst
Original file line number Diff line number Diff line change
Expand Up @@ -962,6 +962,16 @@ With this parameter, the server serves the specified directory, by default it
uses the current working directory.
(Contributed by Stéphane Wirtel and Julien Palard in :issue:`28707`.)

Support HTTP compression : if the user agent sends a request with header
``Accept-Encoding`` including gzip, and if the content type guessed from the
file extension is in the attribute ``compressed_types`` of
:class:`~http.server.SimpleHTTPRequestHandler`, set the response header
``Content-Encoding`` to "gzip", set ``Content-Length`` to the length of the
gzipped content, and return a file object with the gzipped content.
(Contributed by Pierre Quentel in :issue:`30576`.)

hmac
=======
The new :class:`ThreadingHTTPServer <http.server.ThreadingHTTPServer>` class
uses threads to handle requests using :class:`~socketserver.ThreadingMixin`.
It is used when ``http.server`` is run with ``-m``.
Expand Down Expand Up @@ -1085,6 +1095,7 @@ configuration passed to :func:`logging.config.fileConfig`.


math

----

The new :func:`math.remainder` function implements the IEEE 754-style remainder
Expand All @@ -1099,7 +1110,10 @@ The MIME type of .bmp has been changed from ``'image/x-ms-bmp'`` to
(Contributed by Nitish Chandra in :issue:`22589`.)


locale
=======
msilib

------

The new :meth:`Database.Close() <msilib.Database.Close>` method can be used
Expand Down
23 changes: 12 additions & 11 deletions Lib/cgi.py
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ def parse(fp=None, environ=os.environ, keep_blank_values=0, strict_parsing=0):

# field keys and values (except for files) are returned as strings
# an encoding is required to decode the bytes read from self.fp
if hasattr(fp,'encoding'):
if hasattr(fp, 'encoding'):
encoding = fp.encoding
else:
encoding = 'latin-1'
Expand Down Expand Up @@ -202,7 +202,7 @@ def parse_multipart(fp, pdict, encoding="utf-8", errors="replace"):
headers.set_type(ctype)
headers['Content-Length'] = pdict['CONTENT-LENGTH']
fs = FieldStorage(fp, headers=headers, encoding=encoding, errors=errors,
environ={'REQUEST_METHOD': 'POST'})
environ={'REQUEST_METHOD': 'POST'})
return {k: fs.getlist(k) for k in fs}

def _parseparam(s):
Expand Down Expand Up @@ -259,7 +259,6 @@ def __init__(self, name, value):
"""Constructor from field name and value."""
self.name = name
self.value = value
# self.file = StringIO(value)

def __repr__(self):
"""Return printable representation."""
Expand Down Expand Up @@ -456,7 +455,7 @@ def __init__(self, fp=None, headers=None, outerboundary=b'',
if maxlen and clen > maxlen:
raise ValueError('Maximum content length exceeded')
self.length = clen
if self.limit is None and clen:
if self.limit is None and clen >= 0:
self.limit = clen

self.list = self.file = None
Expand Down Expand Up @@ -633,8 +632,10 @@ def read_multi(self, environ, keep_blank_values, strict_parsing):
if 'content-length' in headers:
del headers['content-length']

limit = None if self.limit is None \
else self.limit - self.bytes_read
part = klass(self.fp, headers, ib, environ, keep_blank_values,
strict_parsing,self.limit-self.bytes_read,
strict_parsing, limit,
self.encoding, self.errors)
self.bytes_read += part.bytes_read
self.list.append(part)
Expand All @@ -651,7 +652,7 @@ def read_single(self):
self.read_lines()
self.file.seek(0)

bufsize = 8*1024 # I/O buffering size for copy to file
bufsize = 8 * 1024 # I/O buffering size for copy to file

def read_binary(self):
"""Internal: read binary data."""
Expand Down Expand Up @@ -699,7 +700,7 @@ def __write(self, line):
def read_lines_to_eof(self):
"""Internal: read lines until EOF."""
while 1:
line = self.fp.readline(1<<16) # bytes
line = self.fp.readline(1 << 16) # bytes
self.bytes_read += len(line)
if not line:
self.done = -1
Expand All @@ -717,9 +718,9 @@ def read_lines_to_outerboundary(self):
last_line_lfend = True
_read = 0
while 1:
if _read >= self.limit:
if self.limit is not None and _read >= self.limit:
break
line = self.fp.readline(1<<16) # bytes
line = self.fp.readline(1 << 16) # bytes
self.bytes_read += len(line)
_read += len(line)
if not line:
Expand Down Expand Up @@ -763,7 +764,7 @@ def skip_lines(self):
last_boundary = next_boundary + b"--"
last_line_lfend = True
while True:
line = self.fp.readline(1<<16)
line = self.fp.readline(1 << 16)
self.bytes_read += len(line)
if not line:
self.done = -1
Expand Down Expand Up @@ -805,7 +806,7 @@ def make_file(self):
return tempfile.TemporaryFile("wb+")
else:
return tempfile.TemporaryFile("w+",
encoding=self.encoding, newline = '\n')
encoding=self.encoding, newline='\n')


# Test/debug code
Expand Down
Loading