Skip to content

bpo-39104: Fix hanging ProcessPoolExecutor on shutdown nowait with pi…#252

Merged
sthagen merged 1 commit intosthagen:masterfrom
python:master
Feb 16, 2020
Merged

bpo-39104: Fix hanging ProcessPoolExecutor on shutdown nowait with pi…#252
sthagen merged 1 commit intosthagen:masterfrom
python:master

Conversation

@sthagen
Copy link
Copy Markdown
Owner

@sthagen sthagen commented Feb 16, 2020

…ckling failure (pythonGH-17670)

As reported initially by @rad-pat in python#6084, the following script causes a deadlock.

from concurrent.futures import ProcessPoolExecutor


class ObjectWithPickleError():
    """Triggers a RuntimeError when sending job to the workers"""

    def __reduce__(self):
        raise RuntimeError()


if __name__ == "__main__":
    e = ProcessPoolExecutor()
    f = e.submit(id, ObjectWithPickleError())
    e.shutdown(wait=False)
    f.result()  # Deadlock on get

This is caused by the fact that the main process is closing communication channels that might be necessary to the queue_management_thread later. To avoid this, this PR let the queue_management_thread manage all the closing.

https://bugs.python.org/issue39104

Automerge-Triggered-By: @pitrou

…ckling failure (GH-17670)

As reported initially by @rad-pat in #6084, the following script causes a deadlock.

```
from concurrent.futures import ProcessPoolExecutor


class ObjectWithPickleError():
    """Triggers a RuntimeError when sending job to the workers"""

    def __reduce__(self):
        raise RuntimeError()


if __name__ == "__main__":
    e = ProcessPoolExecutor()
    f = e.submit(id, ObjectWithPickleError())
    e.shutdown(wait=False)
    f.result()  # Deadlock on get
```

This is caused by the fact that the main process is closing communication channels that might be necessary to the `queue_management_thread` later. To avoid this, this PR let the `queue_management_thread` manage all the closing.



https://bugs.python.org/issue39104



Automerge-Triggered-By: @pitrou
@sthagen sthagen merged commit ad2c48a into sthagen:master Feb 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants