Skip to content

Commit 3a17e21

Browse files
committed
merge from 3.4 - clean up the subprocess docs warning-soup and
s/Unix/POSIX/.
2 parents 384a3ea + 8e0aa05 commit 3a17e21

1 file changed

Lines changed: 62 additions & 93 deletions

File tree

‎Doc/library/subprocess.rst‎

Lines changed: 62 additions & 93 deletions
Original file line numberDiff line numberDiff line change
@@ -54,18 +54,12 @@ use cases, the underlying :class:`Popen` interface can be used directly.
5454
>>> subprocess.call("exit 1", shell=True)
5555
1
5656

57-
.. warning::
58-
59-
Invoking the system shell with ``shell=True`` can be a security hazard
60-
if combined with untrusted input. See the warning under
61-
:ref:`frequently-used-arguments` for details.
62-
6357
.. note::
6458

65-
Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function. As
66-
the pipes are not being read in the current process, the child
67-
process may block if it generates enough output to a pipe to fill up
68-
the OS pipe buffer.
59+
Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this
60+
function. The child process will block if it generates enough
61+
output to a pipe to fill up the OS pipe buffer as the pipes are
62+
not being read from.
6963

7064
.. versionchanged:: 3.3
7165
*timeout* was added.
@@ -99,18 +93,12 @@ use cases, the underlying :class:`Popen` interface can be used directly.
9993
...
10094
subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1
10195

102-
.. warning::
103-
104-
Invoking the system shell with ``shell=True`` can be a security hazard
105-
if combined with untrusted input. See the warning under
106-
:ref:`frequently-used-arguments` for details.
107-
10896
.. note::
10997

110-
Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function. As
111-
the pipes are not being read in the current process, the child
112-
process may block if it generates enough output to a pipe to fill up
113-
the OS pipe buffer.
98+
Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this
99+
function. The child process will block if it generates enough
100+
output to a pipe to fill up the OS pipe buffer as the pipes are
101+
not being read from.
114102

115103
.. versionchanged:: 3.3
116104
*timeout* was added.
@@ -177,17 +165,12 @@ use cases, the underlying :class:`Popen` interface can be used directly.
177165
... shell=True)
178166
'ls: non_existent_file: No such file or directory\n'
179167

180-
.. warning::
181-
182-
Invoking the system shell with ``shell=True`` can be a security hazard
183-
if combined with untrusted input. See the warning under
184-
:ref:`frequently-used-arguments` for details.
185-
186168
.. note::
187169

188-
Do not use ``stderr=PIPE`` with this function. As the pipe is not being
189-
read in the current process, the child process may block if it
190-
generates enough output to the pipe to fill up the OS pipe buffer.
170+
Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this
171+
function. The child process will block if it generates enough
172+
output to a pipe to fill up the OS pipe buffer as the pipes are
173+
not being read from.
191174

192175
.. versionadded:: 3.1
193176

@@ -210,7 +193,7 @@ use cases, the underlying :class:`Popen` interface can be used directly.
210193

211194
Special value that can be used as the *stdin*, *stdout* or *stderr* argument
212195
to :class:`Popen` and indicates that a pipe to the standard stream should be
213-
opened.
196+
opened. Most useful with :meth:`Popen.communicate`.
214197

215198

216199
.. data:: STDOUT
@@ -336,28 +319,9 @@ default values. The arguments that are most commonly needed are:
336319
instead of ``locale.getpreferredencoding()``. See the
337320
:class:`io.TextIOWrapper` class for more information on this change.
338321

339-
.. warning::
340-
341-
Executing shell commands that incorporate unsanitized input from an
342-
untrusted source makes a program vulnerable to `shell injection
343-
<http://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_,
344-
a serious security flaw which can result in arbitrary command execution.
345-
For this reason, the use of ``shell=True`` is **strongly discouraged**
346-
in cases where the command string is constructed from external input::
347-
348-
>>> from subprocess import call
349-
>>> filename = input("What file would you like to display?\n")
350-
What file would you like to display?
351-
non_existent; rm -rf / #
352-
>>> call("cat " + filename, shell=True) # Uh-oh. This will end badly...
353-
354-
``shell=False`` disables all shell based features, but does not suffer
355-
from this vulnerability; see the Note in the :class:`Popen` constructor
356-
documentation for helpful hints in getting ``shell=False`` to work.
322+
.. note::
357323

358-
When using ``shell=True``, :func:`shlex.quote` can be used to properly
359-
escape whitespace and shell metacharacters in strings that are going to
360-
be used to construct shell commands.
324+
Read the `Security Considerations`_ section before using ``shell=True``.
361325

362326
These options, along with all of the other options, are described in more
363327
detail in the :class:`Popen` constructor documentation.
@@ -378,7 +342,7 @@ functions.
378342
startupinfo=None, creationflags=0, restore_signals=True, \
379343
start_new_session=False, pass_fds=())
380344

381-
Execute a child program in a new process. On Unix, the class uses
345+
Execute a child program in a new process. On POSIX, the class uses
382346
:meth:`os.execvp`-like behavior to execute the child program. On Windows,
383347
the class uses the Windows ``CreateProcess()`` function. The arguments to
384348
:class:`Popen` are as follows.
@@ -390,7 +354,7 @@ functions.
390354
arguments for additional differences from the default behavior. Unless
391355
otherwise stated, it is recommended to pass *args* as a sequence.
392356

393-
On Unix, if *args* is a string, the string is interpreted as the name or
357+
On POSIX, if *args* is a string, the string is interpreted as the name or
394358
path of the program to execute. However, this can only be done if not
395359
passing arguments to the program.
396360

@@ -421,7 +385,7 @@ functions.
421385
the shell as the program to execute. If *shell* is *True*, it is
422386
recommended to pass *args* as a string rather than as a sequence.
423387

424-
On Unix with ``shell=True``, the shell defaults to :file:`/bin/sh`. If
388+
On POSIX with ``shell=True``, the shell defaults to :file:`/bin/sh`. If
425389
*args* is a string, the string specifies the command
426390
to execute through the shell. This means that the string must be
427391
formatted exactly as it would be when typed at the shell prompt. This
@@ -438,11 +402,9 @@ functions.
438402
into the shell (e.g. :command:`dir` or :command:`copy`). You do not need
439403
``shell=True`` to run a batch file or console-based executable.
440404

441-
.. warning::
405+
.. note::
442406

443-
Passing ``shell=True`` can be a security hazard if combined with
444-
untrusted input. See the warning under :ref:`frequently-used-arguments`
445-
for details.
407+
Read the `Security Considerations`_ section before using ``shell=True``.
446408

447409
*bufsize* will be supplied as the corresponding argument to the :func:`open`
448410
function when creating the stdin/stdout/stderr pipe file objects: :const:`0`
@@ -463,9 +425,9 @@ functions.
463425
program to execute specified by *args*. However, the original *args* is
464426
still passed to the program. Most programs treat the program specified
465427
by *args* as the command name, which can then be different from the program
466-
actually executed. On Unix, the *args* name
428+
actually executed. On POSIX, the *args* name
467429
becomes the display name for the executable in utilities such as
468-
:program:`ps`. If ``shell=True``, on Unix the *executable* argument
430+
:program:`ps`. If ``shell=True``, on POSIX the *executable* argument
469431
specifies a replacement shell for the default :file:`/bin/sh`.
470432

471433
*stdin*, *stdout* and *stderr* specify the executed program's standard input,
@@ -481,7 +443,7 @@ functions.
481443

482444
If *preexec_fn* is set to a callable object, this object will be called in the
483445
child process just before the child is executed.
484-
(Unix only)
446+
(POSIX only)
485447

486448
.. warning::
487449

@@ -499,8 +461,8 @@ functions.
499461
common use of *preexec_fn* to call os.setsid() in the child.
500462

501463
If *close_fds* is true, all file descriptors except :const:`0`, :const:`1` and
502-
:const:`2` will be closed before the child process is executed. (Unix only).
503-
The default varies by platform: Always true on Unix. On Windows it is
464+
:const:`2` will be closed before the child process is executed. (POSIX only).
465+
The default varies by platform: Always true on POSIX. On Windows it is
504466
true when *stdin*/*stdout*/*stderr* are :const:`None`, false otherwise.
505467
On Windows, if *close_fds* is true then no handles will be inherited by the
506468
child process. Note that on Windows, you cannot set *close_fds* to true and
@@ -512,7 +474,7 @@ functions.
512474

513475
*pass_fds* is an optional sequence of file descriptors to keep open
514476
between the parent and child. Providing any *pass_fds* forces
515-
*close_fds* to be :const:`True`. (Unix only)
477+
*close_fds* to be :const:`True`. (POSIX only)
516478

517479
.. versionadded:: 3.2
518480
The *pass_fds* parameter was added.
@@ -525,13 +487,13 @@ functions.
525487
If *restore_signals* is true (the default) all signals that Python has set to
526488
SIG_IGN are restored to SIG_DFL in the child process before the exec.
527489
Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals.
528-
(Unix only)
490+
(POSIX only)
529491

530492
.. versionchanged:: 3.2
531493
*restore_signals* was added.
532494

533495
If *start_new_session* is true the setsid() system call will be made in the
534-
child process prior to the execution of the subprocess. (Unix only)
496+
child process prior to the execution of the subprocess. (POSIX only)
535497

536498
.. versionchanged:: 3.2
537499
*start_new_session* was added.
@@ -598,14 +560,21 @@ Exceptions defined in this module all inherit from :exc:`SubprocessError`.
598560
The :exc:`SubprocessError` base class was added.
599561

600562

601-
Security
602-
^^^^^^^^
563+
Security Considerations
564+
-----------------------
565+
566+
Unlike some other popen functions, this implementation will never
567+
implicitly call a system shell. This means that all characters,
568+
including shell metacharacters, can safely be passed to child processes.
569+
If the shell is invoked explicitly, via ``shell=True``, it is the application's
570+
responsibility to ensure that all whitespace and metacharacters are
571+
quoted appropriately to avoid
572+
`shell injection <http://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_
573+
vulnerabilities.
603574

604-
Unlike some other popen functions, this implementation will never call a
605-
system shell implicitly. This means that all characters, including shell
606-
metacharacters, can safely be passed to child processes. Obviously, if the
607-
shell is invoked explicitly, then it is the application's responsibility to
608-
ensure that all whitespace and metacharacters are quoted appropriately.
575+
When using ``shell=True``, the :func:`shlex.quote` function can be
576+
used to properly escape whitespace and shell metacharacters in strings
577+
that are going to be used to construct shell commands.
609578

610579

611580
Popen Objects
@@ -629,27 +598,27 @@ Instances of the :class:`Popen` class have the following methods:
629598
:exc:`TimeoutExpired` exception. It is safe to catch this exception and
630599
retry the wait.
631600

601+
.. note::
602+
603+
This will deadlock when using ``stdout=PIPE`` or ``stderr=PIPE``
604+
and the child process generates enough output to a pipe such that
605+
it blocks waiting for the OS pipe buffer to accept more data.
606+
Use :meth:`Popen.communicate` when using pipes to avoid that.
607+
632608
.. note::
633609

634610
The function is implemented using a busy loop (non-blocking call and
635611
short sleeps). Use the :mod:`asyncio` module for an asynchronous wait:
636612
see :class:`asyncio.create_subprocess_exec`.
637613

638-
.. warning::
639-
640-
This will deadlock when using ``stdout=PIPE`` and/or
641-
``stderr=PIPE`` and the child process generates enough output to
642-
a pipe such that it blocks waiting for the OS pipe buffer to
643-
accept more data. Use :meth:`communicate` to avoid that.
644-
645614
.. versionchanged:: 3.3
646615
*timeout* was added.
647616

648617
.. deprecated:: 3.4
649618

650-
Do not use the undocumented *endtime* parameter. It is was
651-
unintentionally exposed in 3.3 but was intended to be private
652-
for internal use. Use *timeout* instead.
619+
Do not use the *endtime* parameter. It is was unintentionally
620+
exposed in 3.3 but was left undocumented as it was intended to be
621+
private for internal use. Use *timeout* instead.
653622

654623
.. method:: Popen.communicate(input=None, timeout=None)
655624

@@ -717,13 +686,6 @@ Instances of the :class:`Popen` class have the following methods:
717686

718687
The following attributes are also available:
719688

720-
.. warning::
721-
722-
Use :meth:`~Popen.communicate` rather than :attr:`.stdin.write <Popen.stdin>`,
723-
:attr:`.stdout.read <Popen.stdout>` or :attr:`.stderr.read <Popen.stderr>` to avoid
724-
deadlocks due to any of the other OS pipe buffers filling up and blocking the
725-
child process.
726-
727689
.. attribute:: Popen.args
728690

729691
The *args* argument as it was passed to :class:`Popen` -- a
@@ -757,6 +719,13 @@ The following attributes are also available:
757719
``True``, the stream is a text stream, otherwise it is a byte stream. If the
758720
*stderr* argument was not :data:`PIPE`, this attribute is ``None``.
759721

722+
.. warning::
723+
724+
Use :meth:`~Popen.communicate` rather than :attr:`.stdin.write <Popen.stdin>`,
725+
:attr:`.stdout.read <Popen.stdout>` or :attr:`.stderr.read <Popen.stderr>` to avoid
726+
deadlocks due to any of the other OS pipe buffers filling up and blocking the
727+
child process.
728+
760729

761730
.. attribute:: Popen.pid
762731

@@ -773,7 +742,7 @@ The following attributes are also available:
773742
hasn't terminated yet.
774743

775744
A negative value ``-N`` indicates that the child was terminated by signal
776-
``N`` (Unix only).
745+
``N`` (POSIX only).
777746

778747

779748
Windows Popen Helpers
@@ -1098,7 +1067,7 @@ handling consistency are valid for these functions.
10981067
>>> subprocess.getstatusoutput('/bin/junk')
10991068
(256, 'sh: /bin/junk: not found')
11001069

1101-
Availability: Unix & Windows
1070+
Availability: POSIX & Windows
11021071

11031072
.. versionchanged:: 3.3.4
11041073
Windows support added
@@ -1114,7 +1083,7 @@ handling consistency are valid for these functions.
11141083
>>> subprocess.getoutput('ls /bin/ls')
11151084
'/bin/ls'
11161085

1117-
Availability: Unix & Windows
1086+
Availability: POSIX & Windows
11181087

11191088
.. versionchanged:: 3.3.4
11201089
Windows support added

0 commit comments

Comments
 (0)