Skip to content

[FLINK-33186][checkpoint/tests] Fix flakiness in CheckpointAfterAllTasksFinishedITCase.testRestoreAfterSomeTasksFinished#28800

Open
Efrat19 wants to merge 1 commit into
apache:masterfrom
Efrat19:FLINK-33186
Open

[FLINK-33186][checkpoint/tests] Fix flakiness in CheckpointAfterAllTasksFinishedITCase.testRestoreAfterSomeTasksFinished#28800
Efrat19 wants to merge 1 commit into
apache:masterfrom
Efrat19:FLINK-33186

Conversation

@Efrat19

@Efrat19 Efrat19 commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

What is the purpose of the change

I traced the flakiness to task finished after the checkpoint plan was calculated but before triggerCheckpoint. The task is already removed from TaskExecutor#taskSlotTable So TaskExecutor#triggerCheckpoint fails the checkpoint with error 'TaskManager received a checkpoint request for unknown task <>'.

Production systems are expected to tolerate an occasional checkpoint failure So we can address the flakiness without changes this known limitation for tasks finished mid checkpoint.

the test provisions 2 sources passA that finishes quickly and passB that blocks forever and non-deterministically waits for at least one of passA subtasks to finish, which introduces the race.
The fix here makes CheckpointAfterAllTasksFinishedITCase.testRestoreAfterSomeTasksFinished wait for all subtasks of passA to finish before triggering savepoint, to avoid the race but still achieve the test goal with passB subtasks still blocking.

Brief change log

Make CheckpointAfterAllTasksFinishedITCase.testRestoreAfterSomeTasksFinished wait for all subtasks of passA to finish before triggering savepoint, to avoid the race but still achieve the test goal with passB subtasks still blocking.

Verifying this change

This change is a trivial rework / code cleanup without any test coverage.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable
Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Co-Generated-by: cloud-sonnet-5

…sksFinishedITCase.testRestoreAfterSomeTasksFinished

I traced the flakiness to task finished after the checkpoint plan was calculated
but before triggerCheckpoint. The task is already removed from TaskExecutor#taskSlotTable
So TaskExecutor#triggerCheckpoint fails the checkpoint with error
'TaskManager received a checkpoint request for unknown task <>'.

Production systems are expected to tolerate an occasional checkpoint failure
So we can address the flakiness without changes this known limitation for
tasks finished mid checkpoint.

the test provisions 2 sources passA that finishes quickly and passB that blocks forever
and non-deterministically waits for at least one of passA subtasks to finish, which introduces
the race.
The fix here makes CheckpointAfterAllTasksFinishedITCase.testRestoreAfterSomeTasksFinished
wait for all subtasks of passA to finish before triggering savepoint, to avoid the race
but still achieve the test goal with passB subtasks still locking.
@flinkbot

flinkbot commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants