Skip to content

Failing Dask Distributed Test Due to Double Closing of Cluster #1115

@seanlaw

Description

@seanlaw

This test has been failing frequently and it was suggested:

The job failed due to an error during the teardown of the test test_aampdisted[T_A1-T_B1] in tests/test_aampdist.py. The error trace shows that a CancelledError occurred in asyncio/distributed code when cleaning up the Dask cluster:

The fixture dask_cluster in tests/test_aampdist.py (see lines 11–21 in ref 27e1eed) sets up and closes a LocalCluster for Dask tests.
The test test_aampdisted uses this fixture and the dask.distributed.Client context manager.
This pattern is leading to teardown errors, usually because the Client context manager closes the cluster automatically, and then pytest tries to close it again in the fixture finalizer, causing the CancelledError.

Solution:
Modify dask_cluster so it yields the cluster’s address, and construct the Client outside the fixture to avoid double-closing.

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingenhancementNew feature or requesttesting

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions