Repository navigation
iscoroutinefunction broken with cython - allow tagging of functions as async? #82406
Description
| BPO | 38225 |
|---|---|
| Nosy | @scoder, @asvetlov, @1st1, @eamanu, @dhiltonp, @fornellas |
| PRs |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2019-09-19.18:38:39.794>
labels = ['type-feature', '3.10', 'expert-asyncio']
title = 'iscoroutinefunction broken with cython - allow tagging of functions as async?'
updated_at = <Date 2020-09-22.11:36:05.940>
user = 'https://github.com/dhiltonp'bugs.python.org fields:
activity = <Date 2020-09-22.11:36:05.940>
actor = 'scoder'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['asyncio']
creation = <Date 2019-09-19.18:38:39.794>
creator = 'dhiltonp'
dependencies = []
files = []
hgrepos = []
issue_num = 38225
keywords = ['patch']
message_count = 10.0
messages = ['352812', '352817', '352832', '352843', '352906', '352940', '358224', '358842', '358843', '377313']
nosy_count = 6.0
nosy_names = ['scoder', 'asvetlov', 'yselivanov', 'eamanu', 'dhiltonp', 'fornellas']
pr_nums = ['16292']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue38225'
versions = ['Python 3.10']Linked PRs
- gh-82406: document duck-type nature of inspect.is*function #94987
- [3.13] gh-82406: Add 3.10.6 change notes for inspect.is*function (GH-94987) #156880
- [3.14] gh-82406: Add 3.10.6 change notes for inspect.is*function (GH-94987) #156881
- [3.15] gh-82406: Add 3.10.6 change notes for inspect.is*function (GH-94987) #156882
Activity
dhiltonp commentedon Sep 19, 2019
If a python piece of code imports cython code with async defs,
asyncio.iscoroutinefunctioncannot determine that the code is async.cython/cython#2273 (comment)
scoder is open to marking async defs so that they can be identified, just like
asyncio.coroutine:cpython/Lib/asyncio/coroutines.py
Line 156 in ae239f6
However, that is an internal interface and
@coroutineis deprecated.--------------
Can we have some official way of marking functions as async that will not be deprecated?
The easiest would be for
asyncio.iscoroutinefunctionto look for_is_coroutine = True, and promise to keep looking for that value.This would also allow for functools.partial to easily mark that it's returning an async function, which some people seem to care about.
asvetlov commentedon Sep 19, 2019
I think we need better name than just
_is_coroutine.All async function properties as dunder named, the new official name should follow this convention as well
scoder commentedon Sep 20, 2019
My usual first reaction is: "if you need to find out whether the return value of a callable will be an Awaitable or not, without calling it, then you're probably doing something wrong in your design".
However,
a) there is code that tries this already (and it falls short in various ways while trying)
b) asyncio has a function
iscoroutinefunctionwhich seems to fulfil this need but does not achieve it (because not everything that returns an Awaitable is a "coroutine function")c) asyncio has an internal protocol for marking things as "is a coroutine", which comes close to but isn't "returns an Awaitable when called"
So – should there be an official protocol for marking callables as returning an Awaitable? Should we look at annotations for that? Anything else? Or do we consider this intention inherently flawed?
asvetlov commentedon Sep 20, 2019
I think
func.__awaitable__ = Truecan serve all possible situations.We need it for async mocks (now the library use flawless
_is_coroutineapproach.__awaitable__can also cover cases where a regular function returns awaitable object or there is a class withdef __await__(self)method,functools.partial-- all of them are awaitable.Async functions can be modified to provide the property out of the box.
Regarding the question "is a check for awaitableness a good design or not"?
I agree that a check for awaitableness is useless *just before* calling the function.
There is another very useful case where the check is important: registering a callback for later usage.
For example, we have a web server framework. It is built around a mapping of URL paths to async functions, e.g. path('/hello', say_hello) in Django style. It's crucial to check if say_hello() is an async function *on the application configuration stage*, not on viewing the particular http://localhost/hello page in a browser.
So, I think an official protocol makes a lot of sense, let's implement it in 3.9
1st1 commentedon Sep 20, 2019
Well, this seems to be a regression. IIRC Cython used to emulate a Python code object (code) for functions it compiled along with the appropriate CO_COROUTINE flag set. Wasn't that the case?
scoder commentedon Sep 21, 2019
No, it never did that, for safety reasons. It's difficult to say if enabling these flags is the right thing to do, because it's unclear what assumptions code that tests for them would make. In CPython itself, there do not seem to be any "excessive" assumptions specific to that flag – also because Cython functions are not Python functions, and thus the flag will never be looked at:
cpython/Lib/inspect.py
Lines 178 to 180 in 5b9ff7a
Thus, setting the CO_COROUTINE and CO_ASYNC_GENERATOR code flags has no effect for Cython functions.
fornellas commentedon Dec 10, 2019
It is worth noting that test frameworks can greatly benefit from iscoroutinefunction to work.
I'm the main author of TestSlide, which provides more strict mocking for Python. I recently added async support, so we can detect bugs such as configuring a sync mock for something that is async (https://testslide.readthedocs.io/en/2.0.2/strict_mock/index.html#coroutine-functions-async-def). It works just fine, as long as iscoroutinefunction works, which is not the case for async Cython (a big chunk of TestSlide's use cases).
+1 an Andrew's func.awaitable idea: interpreter can provide that for
async defand one can also do that for anything that implements call to communicate that the callable is async.scoder commentedon Dec 24, 2019
FWIW, it seems reasonable to have a protocol for this.
asvetlov commentedon Dec 24, 2019
I'd like to hear Yuri's opinion for the idea of adding an attribute.
Another question is the name: I have proposed __awaitable__ but maybe __async__ is slightly better?
scoder commentedon Sep 22, 2020
FYI, cython/cython#3427 has been merged into Cython. In Cython 3.0, compiled coroutines will disguise as non-compiled coroutines, from the PoV of asyncio.
Restricting this ticket to Py3.10 since we're rather discussing a new feature now.
1 remaining item
graingert commentedon Jul 18, 2022
@ambv this is fixed in 4261b6b
pythongh-82406: document duck-type nature of inspect.is*function
gh-82406: Add 3.10.6 change notes for inspect.is*function (GH-94987)
[3.14] gh-82406: Add 3.10.6 change notes for inspect.is*function (GH-…
[3.13] gh-82406: Add 3.10.6 change notes for inspect.is*function (GH-…
pythongh-82406: Add 3.10.6 change notes for inspect.is*function (pyth…
[3.15] gh-82406: Add 3.10.6 change notes for inspect.is*function (GH-…