asyncio.as_completed() leaves stale awaited_by edges when the loop exits early

Bug report

I was trying to make some tools for fetching websites for my AI tools, when I noticed something odd in the call graph. I fetch a few hosts and use as_completed() to take the first result, bailing out if one fails. After that, the tasks that are still running keep showing the finished caller as "Awaited by".

import asyncio

async def fetch(host):
    reader, writer = await asyncio.open_connection(host, 80)
    writer.write(f"GET / HTTP/1.1\r\nHost: {host}\r\nConnection: close\r\n\r\n".encode())
    await writer.drain()
    body = await reader.read()
    writer.close()
    await writer.wait_closed()
    return host, body

async def fetch_first(tasks):
    for fut in asyncio.as_completed(tasks):
        try:
            return await fut
        except OSError as e:
            print("failed:", e)
            break

async def main():
    good = asyncio.create_task(fetch("example.com"), name="fetch example.com")
    bad = asyncio.create_task(fetch("doesnotexist.invalid"), name="fetch doesnotexist.invalid")
    w = asyncio.create_task(fetch_first([good, bad]), name="fetch_first")
    await w
    print("fetch_first done:", w.done())
    print(asyncio.format_call_graph(good))
    await good

asyncio.run(main())

Output:

failed: [Errno 11001] getaddrinfo failed
fetch_first done: True
* Task(name='fetch example.com', id=0x...)
  + Call stack:
  |   ...
  |   File 'repro.py', line 4, in async fetch()
  + Awaited by:
    * Task(name='fetch_first', id=0x...)

fetch_first is already done but is still listed as awaiting fetch example.com. The edge only goes away once that task finishes. gather(), wait(), wait_for() and TaskGroup don't leave edges like this in the same situation, only as_completed() does.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Windows

I can reproduce this on current main (3.16.0a1+dev, Linux). Here is a smaller version that needs no network:

import asyncio

async def slow():
    await asyncio.sleep(0.5)

async def fails():
    await asyncio.sleep(0.01)
    raise OSError("boom")

async def fetch_first(tasks):
    for fut in asyncio.as_completed(tasks):
        try:
            return await fut
        except OSError:
            break

async def main():
    good = asyncio.create_task(slow(), name="slow")
    bad = asyncio.create_task(fails(), name="fails")
    w = asyncio.create_task(fetch_first([good, bad]), name="fetch_first")
    await w
    print(asyncio.format_call_graph(good))  # still shows fetch_first under "Awaited by"
    await good

asyncio.run(main())

Cause: _AsCompletedIterator.__init__ (Lib/asyncio/tasks.py) calls future_add_to_awaited_by(f, cur_task) for every pending future. The edge is only removed when that future completes (_handle_completion) or on timeout (_handle_timeout). If the caller stops iterating early, nothing removes it.

I would like to work on a fix. Before opening a PR I wanted to ask which approach you prefer. I prototyped the first one and found a performance problem:

Add the edges only while the caller is actually waiting, and remove them in a finally in _wait_for_one(). This is deterministic and fixes the repro above. The cost is that each wait that suspends adds and removes edges for every pending future. In a worst-case benchmark (a consumer that suspends on every item) 10,000 pending futures went from about 58 ms to about 3.4 s, so this is quadratic. I don't think this is acceptable as is.
Discard the remaining edges when the calling task finishes, with a done callback. This is cheap, but task done-callbacks run after the awaiting task resumes, so a script like the one above would still see the stale edge for one loop iteration.
Discard the edges when the iterator is garbage-collected. This needs weak references in the done callbacks and the timeout handle, so it's a bigger change, and it depends on reference counting (so PyPy may behave differently).
Skip finished tasks in the awaited-by output of capture_call_graph() / format_call_graph(). This is cheap and fixes the reported symptom, but it hides the stale edge instead of removing it, and external tools that read the set directly (for example the remote debugging code) would still see it.

Is the stale edge considered a bug here, and if so, which of these (or another approach) would you accept? I'll wait for your feedback before opening a PR.

Navigation Menu

Repository navigation

asyncio.as_completed() leaves stale awaited_by edges when the loop exits early #161766

Bug report

CPython versions tested on:

Operating systems tested on:

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

asyncio.as_completed() leaves stale awaited_by edges when the loop exits early #161766

Description

Bug report

CPython versions tested on:

Operating systems tested on:

Activity

parthkale275 commented on Nov 10, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions