gh-106461: typing: Consolidate docs on Callable (#106462)

AlexWaygood · web-flow · commit ca8b55c7f54b · 2023-07-09T21:46:15.000+01:00
diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst
@@ -258,18 +258,21 @@ See :pep:`484` for more details.
    The performance of calling ``NewType`` has been restored to its level in
    Python 3.9.
 
+.. _annotating-callables:
 
-Callable
-========
+Annotating callable objects
+===========================
 
-Frameworks expecting callback functions of specific signatures might be
-type hinted using ``Callable[[Arg1Type, Arg2Type], ReturnType]``.
+Functions -- or other :term:`callable` objects -- can be annotated using
+:class:`collections.abc.Callable` or :data:`typing.Callable`.
+``Callable[[int], str]`` signifies a function that takes a single parameter
+of type :class:`int` and returns a :class:`str`.
 
 For example:
 
 .. testcode::
 
-   from collections.abc import Callable
+   from collections.abc import Callable, Awaitable
 
    def feeder(get_next_item: Callable[[], str]) -> None:
        ...  # Body
@@ -283,9 +286,49 @@ For example:
 
    callback: Callable[[str], Awaitable[None]] = on_update
 
-It is possible to declare the return type of a callable without specifying
-the call signature by substituting a literal ellipsis
-for the list of arguments in the type hint: ``Callable[..., ReturnType]``.
+The subscription syntax must always be used with exactly two values: the
+argument list and the return type.  The argument list must be a list of types,
+a :class:`ParamSpec`, :data:`Concatenate`, or an ellipsis. The return type must
+be a single type.
+
+If a literal ellipsis ``...`` is given as the argument list, it indicates that
+a callable with any arbitrary parameter list would be acceptable:
+
+.. testcode::
+
+   def concat(x: str, y: str) -> str:
+       return x + y
+
+   x: Callable[..., str]
+   x = str     # OK
+   x = concat  # Also OK
+
+``Callable`` cannot express complex signatures such as functions that take a
+variadic number of arguments, :func:`overloaded functions <overload>`, or
+functions that have keyword-only parameters. However, these signatures can be
+expressed by defining a :class:`Protocol` class with a
+:meth:`~object.__call__` method:
+
+.. testcode::
+
+   from collections.abc import Iterable
+   from typing import Protocol
+
+   class Combiner(Protocol):
+       def __call__(self, *vals: bytes, maxlen: int | None = None) -> list[bytes]: ...
+
+   def batch_proc(data: Iterable[bytes], cb_results: Combiner) -> bytes:
+       for item in data:
+           ...
+
+   def good_cb(*vals: bytes, maxlen: int | None = None) -> list[bytes]:
+       ...
+   def bad_cb(*vals: bytes, maxitems: int | None) -> list[bytes]:
+       ...
+
+   batch_proc([], good_cb)  # OK
+   batch_proc([], bad_cb)   # Error! Argument 2 has incompatible type because of
+                            # different name and kind in the callback
 
 Callables which take other callables as arguments may indicate that their
 parameter types are dependent on each other using :class:`ParamSpec`.
@@ -1043,56 +1086,16 @@ These can be used as types in annotations. They all support subscription using
       Optional can now be written as ``X | None``. See
       :ref:`union type expressions<types-union>`.
 
-.. data:: Callable
-
-   Deprecated alias to :class:`collections.abc.Callable`.
-
-   ``Callable[[int], str]`` signifies a function that takes a single parameter
-   of type :class:`int` and returns a :class:`str`.
-
-   The subscription syntax must always be used with exactly two
-   values: the argument list and the return type.  The argument list
-   must be a list of types, a :class:`ParamSpec`, :data:`Concatenate`,
-   or an ellipsis. The return type must be a single type.
-
-   There is no syntax to indicate optional or keyword arguments;
-   such function types are rarely used as callback types.
-   ``Callable[..., ReturnType]`` (literal ellipsis) can be used to
-   type hint a callable taking any number of arguments and returning
-   ``ReturnType``.  A plain :data:`Callable` is equivalent to
-   ``Callable[..., Any]``, and in turn to
-   :class:`collections.abc.Callable`.
-
-   Callables which take other callables as arguments may indicate that their
-   parameter types are dependent on each other using :class:`ParamSpec`.
-   Additionally, if that callable adds or removes arguments from other
-   callables, the :data:`Concatenate` operator may be used.  They
-   take the form ``Callable[ParamSpecVariable, ReturnType]`` and
-   ``Callable[Concatenate[Arg1Type, Arg2Type, ..., ParamSpecVariable], ReturnType]``
-   respectively.
-
-   .. deprecated:: 3.9
-      :class:`collections.abc.Callable` now supports subscripting (``[]``).
-      See :pep:`585` and :ref:`types-genericalias`.
-
-   .. versionchanged:: 3.10
-      ``Callable`` now supports :class:`ParamSpec` and :data:`Concatenate`.
-      See :pep:`612` for more details.
-
-   .. seealso::
-      The documentation for :class:`ParamSpec` and :class:`Concatenate` provide
-      examples of usage with ``Callable``.
-
 .. data:: Concatenate
 
    Special form for annotating higher-order functions.
 
-   ``Concatenate`` can be used in conjunction with :data:`Callable` and
+   ``Concatenate`` can be used in conjunction with :ref:`Callable <annotating-callables>` and
    :class:`ParamSpec` to annotate a higher-order callable which adds, removes,
    or transforms parameters of another
    callable.  Usage is in the form
    ``Concatenate[Arg1Type, Arg2Type, ..., ParamSpecVariable]``. ``Concatenate``
-   is currently only valid when used as the first argument to a :data:`Callable`.
+   is currently only valid when used as the first argument to a :ref:`Callable <annotating-callables>`.
    The last parameter to ``Concatenate`` must be a :class:`ParamSpec` or
    ellipsis (``...``).
 
@@ -1136,8 +1139,9 @@ These can be used as types in annotations. They all support subscription using
    .. seealso::
 
       * :pep:`612` -- Parameter Specification Variables (the PEP which introduced
-        ``ParamSpec`` and ``Concatenate``).
-      * :class:`ParamSpec` and :class:`Callable`.
+        ``ParamSpec`` and ``Concatenate``)
+      * :class:`ParamSpec`
+      * :ref:`annotating-callables`
 
 .. data:: Literal
 
@@ -1893,8 +1897,9 @@ without the dedicated syntax, as documented below.
 
    .. seealso::
       * :pep:`612` -- Parameter Specification Variables (the PEP which introduced
-        ``ParamSpec`` and ``Concatenate``).
-      * :class:`Callable` and :class:`Concatenate`.
+        ``ParamSpec`` and ``Concatenate``)
+      * :data:`Concatenate`
+      * :ref:`annotating-callables`
 
 .. data:: ParamSpecArgs
 .. data:: ParamSpecKwargs
@@ -2179,7 +2184,7 @@ types.
         methods or attributes, not their type signatures or types.
         For example, :class:`ssl.SSLObject`
         is a class, therefore it passes an :func:`issubclass`
-        check against :data:`Callable`.  However, the
+        check against :ref:`Callable <annotating-callables>`. However, the
         ``ssl.SSLObject.__init__`` method exists only to raise a
         :exc:`TypeError` with a more informative message, therefore making
         it impossible to call (instantiate) :class:`ssl.SSLObject`.
@@ -3533,6 +3538,21 @@ Aliases to other ABCs in :mod:`collections.abc`
       :class:`collections.abc.Iterator` now supports subscripting (``[]``).
       See :pep:`585` and :ref:`types-genericalias`.
 
+.. data:: Callable
+
+   Deprecated alias to :class:`collections.abc.Callable`.
+
+   See :ref:`annotating-callables` for details on how to use
+   :class:`collections.abc.Callable` and ``typing.Callable`` in type annotations.
+
+   .. deprecated:: 3.9
+      :class:`collections.abc.Callable` now supports subscripting (``[]``).
+      See :pep:`585` and :ref:`types-genericalias`.
+
+   .. versionchanged:: 3.10
+      ``Callable`` now supports :class:`ParamSpec` and :data:`Concatenate`.
+      See :pep:`612` for more details.
+
 .. class:: Generator(Iterator[YieldType], Generic[YieldType, SendType, ReturnType])
 
    Deprecated alias to :class:`collections.abc.Generator`.
