Repository navigation
Description
Summary
An activity failure marked failureDetails.isNonRetriable=true is still retried when the Python orchestration uses an explicit retry policy. The retryable-task failure path calculates the next delay without checking the flag.
I reproduced this twice with a real Python Functions host, a DTS emulator, and Azurite. The provider correctly recorded the non-retriable failure in backend history; Python then scheduled a retry timer and another activity attempt.
Related context: azure-functions-durable-extension#3582 and the follow-up on #269. This is a separate Python retry-policy issue, not a recurrence of the shared-host payload-hydration disconnect.
Environment
- Python 3.14.10 on Windows
azure-functions==2.3.0azure-functions-durable==2.0.0rc2durabletask==1.12.0- Functions Core Tools 4.16.0
- Durable Functions host extension 3.15.1
- AzureManaged provider with host-side
payloadStorageEnabled=true,payloadStorageThresholdBytes=131072, and gzip enabled - Python SDK-managed payload externalization disabled
Reproduction
Use an orchestration with an explicit activity retry policy and compensation handling:
@app.orchestration_trigger(context_name="context")
def orchestration(context: df.DurableOrchestrationContext):
try:
result = yield context.call_activity_with_retry(
"consume_payload",
df.RetryOptions(1000, 3),
"x" * 160000,
)
return {"result": result, "compensated": False}
except Exception:
return {"compensated": True}
@app.activity_trigger(input_name="payload")
def consume_payload(payload: str):
return len(payload)- Gate the first activity-input blob download, then delete the committed blob before allowing the download to proceed.
- Use a provider that settles this missing-input delivery with a
TaskFailedwhosefailureDetails.isNonRetriableistrue. Verify that flag in raw backend history. A deterministic SDK regression test can inject the same failure event directly, without blob storage. - Let the Python orchestration process that failure.
RetryOptions is the compatibility wrapper around durabletask.task.RetryPolicy; the underlying retry handling is shared.
Expected
The non-retriable failure should override the retry policy: one activity scheduling, no retry timer, and the exception should reach the orchestration's compensation handler.
Actual
Both runs recorded this sequence in raw backend history:
TaskScheduled(consume_payload)
TaskFailed(failureDetails.isNonRetriable=true)
TimerCreated
TimerFired
TaskScheduled(consume_payload)
TaskCompleted
ExecutionCompleted
There were two activity schedulings and one retry timer, despite the non-retriable flag. In this workload, the second scheduling externalized a new, readable input blob, so the retry succeeded and the orchestration returned compensated=false rather than entering its exception handler.
Without an explicit retry policy, the same provider failure reached the Python exception handler as expected. Unrelated in-flight activities remained isolated and completed normally.
Source evidence
In v1.12.0, the retryable activity failure branch calls compute_next_delay() and schedules a timer whenever a delay is available, without checking event.taskFailed.failureDetails.isNonRetriable.
The installed wheel's worker.py exactly matched the tagged source (Git blob 25010a17af0ed57143f399ece3176829387f08d3). The sub-orchestration retry branch appears to have the same omission, although I did not perform a separate live sub-orchestration retry-policy reproduction.
Suggested regression coverage
- An activity
TaskFailedwithisNonRetriable=truefails the pending retryable task immediately, resumes the orchestrator, and creates neither a retry timer nor another activity attempt. - An otherwise identical failure with the flag unset/false retains existing retry-policy behavior.
- Check the analogous sub-orchestration path.
This should honor an explicit non-retriable failure flag, not classify ordinary transient storage errors as permanent. Provider-side abandonment/redelivery for recoverable hydration failures is a different mechanism.
Activity