Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
b4e4f97
Prototype replay-driven async orchestrators across providers
andystaples Oct 15, 2026
e63cf01
Integrate async prototype with current main and dispatch recovery
andystaples Oct 30, 2026
ee356d7
Add deterministic asyncio scheduling and durable timers to shared replay
andystaples Oct 30, 2026
40b30ac
Preserve exceptional-exit policy for async-generator teardown
andystaples Oct 30, 2026
8c50027
Prove native Pydantic AnyIO execution and cross-provider asyncio paths
andystaples Oct 30, 2026
610a851
Document coordinated async contract across provider entry points
andystaples Oct 30, 2026
e314a69
Cover cancellation action order and Python 3.10 timeout behavior
andystaples Oct 30, 2026
8c050a7
Report unobserved background failures and drain cleanup errors
andystaples Oct 30, 2026
635f5d3
Harden async cleanup and scale completion delivery after review
andystaples Nov 2, 2026
22b30ef
Integrate current main and align async checks with Python 3.11 baseline
andystaples Nov 2, 2026
695bb12
Keep async documentation focused on feature behavior
andystaples Nov 2, 2026
eba4431
Preserve process-control exits across coroutine cleanup failures
andystaples Nov 2, 2026
688c8cb
Harden async callbacks and retain framework-independent coverage
andystaples Nov 2, 2026
01f47ec
Tighten shared async orchestration runtime integration
andystaples Nov 2, 2026
77c09bc
Fix async test-driver isolation and replay result retention
andystaples Nov 2, 2026
2cb9d0d
Merge latest main and preserve unreleased async feature notes
andystaples Nov 3, 2026
9be5c4e
Scope async orchestration support to standard-library asyncio
andystaples Nov 3, 2026
b45d127
Release observed async failure tracebacks during replay
andystaples Nov 3, 2026
0872493
Add cold async orchestration operations with owned waits
andystaples Nov 3, 2026
0c21edd
Merge current main into coroutine orchestration branch
andystaples Nov 10, 2026
da1d131
Make async orchestrators coroutine-first
andystaples Nov 10, 2026
db268a2
Merge main and integrate durable HTTP retries
andystaples Nov 10, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .github/workflows/durabletask.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,31 @@ permissions:
contents: read

jobs:
async-runtime-windows:
strategy:
fail-fast: false
matrix:
python-version: ["3.11", "3.12", "3.13", "3.14", "3.15"]
runs-on: windows-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
with:
python-version: ${{ matrix.python-version }}
- name: Install runtime test dependencies
run: |
python -m pip install --upgrade "pip>=25.1"
python -m pip install --group test -e .
- name: Check asyncio, replay, and dispatch cleanup
run: >-
python -m pytest
tests\durabletask\test_async_context.py
tests\durabletask\test_asyncio_orchestrators.py
tests\durabletask\test_coroutine_orchestrators.py
tests\durabletask\test_orchestration_generator_cleanup.py
tests\durabletask\test_orchestration_dispatch_recovery.py
-q

lint-and-unit-tests:
runs-on: ubuntu-latest
steps:
Expand Down
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,23 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## Unreleased

ADDED

- Added `async def` orchestrators with an automatically supplied
`AsyncOrchestrationContext`, cold coroutine operations, durable asyncio
timers, and supported task-group and gather composition. Generator
orchestrators retain their eager durable handles. `as_task_context()` and
`Task.wait()` enable explicit interoperability with those handles.
- Async event waits support owned cancellation and timeouts, withdrawing
abandoned subscriptions so they cannot consume later events.

> [!WARNING]
> Async cleanup can run at every dispatch, including suspension; it is not
> durable workflow compensation. Converting running orchestrations or changing
> Python versions may alter replay-visible action order. See the
> [async contract](docs/async-orchestrators.md) for supported operations and
> replay and upgrade guidance.

FIXED

- Activity and sub-orchestration `retry_timeout` decisions now use replay-safe
Expand Down
47 changes: 47 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,53 @@ The Azure Functions Durable provider requires Python 3.13 or later.

## Optional Features

### Async Orchestrators

All three packages support `async def` orchestrators and nested async helpers.
Existing generator orchestrators continue to work.

```python
import asyncio

from durabletask import task


async def sequence(ctx: task.AsyncOrchestrationContext, value: int) -> list[int]:
first = await ctx.call_activity("plus_one", input=value, return_type=int)
async with asyncio.TaskGroup() as group:
left = group.create_task(ctx.call_activity("plus_one", input=first, return_type=int))
right = group.create_task(ctx.call_activity("plus_one", input=value, return_type=int))
return [left.result(), right.result()]
```

Register the function with `TaskHubGrpcWorker`, `DurableTaskSchedulerWorker`,
or the Azure Functions `orchestration_trigger` decorator as usual. Functions
also supports single-argument `async def` orchestrators using the compatibility
context and `await context.task_all(...)`. Async orchestrators automatically
receive cold coroutine operations accepted by `create_task` and task groups.
Generator orchestrators keep their eager, reusable durable handles. For
interoperability, `ctx.as_task_context()` exposes the eager context and
`handle.wait()` creates an independently cancellable borrower.

> [!WARNING]
> This is replay-driven await syntax, not general-purpose asyncio execution.
> Compose durable work with `asyncio.gather`, local tasks/task groups, and
> durable `asyncio.sleep` / timeouts within the
> [supported contract](docs/async-orchestrators.md).
> Perform network, storage, and other nondeterministic I/O in activities.
> Completed foreign awaitables and synchronous side effects cannot reliably be
> detected by the replay engine.
> Python `finally` blocks and async context-manager exits can run at ordinary
> dispatch suspension; they are not workflow-lifetime compensation scopes.

An async helper is not a durable sub-orchestration: use
`await ctx.call_sub_orchestrator(...)` for a separate orchestration instance.
An `async def` function containing `yield` is an async generator, not a supported
orchestrator. Converting an existing orchestrator must preserve durable action
order and payloads for replay compatibility.
See the contract for cancellation, per-dispatch async cleanup, unsupported
ordering-sensitive APIs and framework adapters.

### Task Hub Query Filters

Set `OrchestrationQuery.task_hub_names` to filter orchestration queries by one or
Expand Down
11 changes: 11 additions & 0 deletions azure-functions-durable/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

ADDED

- With the corresponding core SDK, native and v1-style `async def`
orchestrators automatically receive async contexts with cold durable operations
and supported asyncio composition. The v1-style context preserves compatibility
argument names and provides async `task_all` and `task_any`. Generator
orchestrators retain their eager behavior. See the
[async contract](../docs/async-orchestrators.md) for dispatch cleanup and
replay/upgrade precautions.
- The `orchestrator_generator_wrapper` test helper accepts direct-await
coroutines. Tests involving asyncio scheduling still require a replay executor
or backend.
- Added opt-in `HttpRetryOptions` on `context.call_http()` for durable initial
request retries with configurable timing and status selection. Empty or omitted
status lists select all non-2xx responses; exhausted retries fail the call.
Expand Down Expand Up @@ -59,6 +69,7 @@ readers before enabling new writers; existing headerless blobs still require
their original compression setting. See the
[core compatibility guidance](../docs/features.md#cross-sdk-compatibility).


## v2.0.0rc4

ADDED
Expand Down
10 changes: 9 additions & 1 deletion azure-functions-durable/MIGRATION_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,10 +102,18 @@ The single-argument orchestrator receives the compatibility
Use this state to validate the infrastructure upgrade before changing
application APIs.

In 2.x, a single-argument `async def` orchestrator instead receives
`AsyncDurableOrchestrationContext`, with the same v1-style argument names and
awaitable durable operations. Generator orchestrators retain their existing
eager task-handle behavior. See the
[async authoring contract](../docs/async-orchestrators.md) before converting
running orchestrations.

## Migrate to native orchestration APIs

Native orchestrators accept the durabletask context and input as separate
arguments:
arguments. Generator functions receive `task.OrchestrationContext`;
`async def` functions receive `task.AsyncOrchestrationContext`:

```python
from typing import Any
Expand Down
25 changes: 25 additions & 0 deletions azure-functions-durable/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,30 @@ their existing page sizes.
so they can be used as dictionary keys or set elements. Construct a new ID
instead of assigning or deleting attributes on an existing ID.

## Async orchestrators

Both authoring styles accept `async def` and `await` without an adapter loop.
Await durable operations and compose branches with `asyncio.gather` or task
groups; `asyncio.sleep` and timeouts use durable timers. Synchronous generator
orchestrators remain supported, including mixed-style sub-orchestrations.

Async orchestrators automatically receive cold coroutine operations accepted by
`create_task` and task groups. Native orchestrators receive
`task.AsyncOrchestrationContext`; single-argument orchestrators receive
`df.AsyncDurableOrchestrationContext`, retaining v1-style argument names and
`await context.task_all(...)` / `await context.task_any(...)`.
`wait_for_external_event()` owns its registration and optional durable timeout,
so an abandoned receive cannot consume the next event. Generator contexts
retain eager, reusable durable handles; async code can access those through
`context.as_task_context()` and use `handle.wait()` to create a borrowed waiter.

The [shared async contract](../docs/async-orchestrators.md) defines the supported
API subset, cancellation, per-dispatch `finally`/async-generator cleanup,
replay/migration rules, and framework adapter responsibilities. Arbitrary I/O,
threads, subprocesses, `asyncio.wait`/`as_completed`, and automatic recording of
ordinary async helpers are not supported. Put nondeterministic work in ordinary
activities, not orchestrators.

## Durable HTTP failure retries

Opt in to durable retries for the **initial** `context.call_http()` request with
Expand Down Expand Up @@ -93,6 +117,7 @@ rather than returning the last selected response. Transport and other activity
failures also use the policy regardless of the status list.

The policy and status list are snapshotted when the request is scheduled.
For async orchestrators, scheduling occurs when the coroutine starts.
Omitting `retry_options` preserves the existing request payload and behavior.
Automatic 202/Location polling remains separate: subsequent GET polls inherit
neither the failure retry policy nor its status classification. Poll HTTP
Expand Down
6 changes: 5 additions & 1 deletion azure-functions-durable/azure/durable_functions/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,10 @@
from .internal.compat.entity_state_response import EntityStateResponse
from .internal.compat.entity_id import EntityId
from .internal.compat.token_source import ManagedIdentityTokenSource, TokenSource
from .internal.compat.orchestration_context import DurableOrchestrationContext
from .internal.compat.orchestration_context import (
AsyncDurableOrchestrationContext,
DurableOrchestrationContext,
)
from .internal.compat.entity_context import DurableEntityContext
from .internal.compat.compat_aliases import (
DurableOrchestrationClient,
Expand All @@ -39,6 +42,7 @@

__all__ = [
"Blueprint",
"AsyncDurableOrchestrationContext",
"DFApp",
"DurableEntityContext",
"DurableFunctionsClient",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
builtin_http_poll_orchestrator,
)
from ..internal.compat.activity import wrap_activity, wrap_activity_payloads
from ..internal.compat.orchestration_context import FunctionsOrchestrator
from ..internal.invocation import preserve_function_source, wrap_invocation
from ..worker import DurableFunctionsWorker

Expand Down Expand Up @@ -169,7 +170,7 @@ def _configure_orchestrator_callable(
wrap: Callable[[Callable[..., Any]], FunctionBuilder],
context_name: str,
input_type: Optional[type] = None
) -> Callable[[task.Orchestrator[Any, Any]], FunctionBuilder]:
) -> Callable[[FunctionsOrchestrator], FunctionBuilder]:
"""Obtain decorator to construct an Orchestrator class from a user-defined Function.
Parameters
Expand All @@ -187,7 +188,7 @@ def _configure_orchestrator_callable(
The function to construct an Orchestrator class from the user-defined Function,
wrapped by the next decorator in the sequence.
"""
def decorator(orchestrator_func: task.Orchestrator[Any, Any]) -> FunctionBuilder:
def decorator(orchestrator_func: FunctionsOrchestrator) -> FunctionBuilder:
# Construct an orchestrator based on the end-user code

if input_type is not None:
Expand Down Expand Up @@ -281,7 +282,7 @@ def decorator(func: Callable[..., Any]) -> FunctionBuilder:
def orchestration_trigger(self, context_name: str,
orchestration: Optional[str] = None,
input_type: Optional[type] = None
) -> Callable[[task.Orchestrator[Any, Any]], FunctionBuilder]:
) -> Callable[[FunctionsOrchestrator], FunctionBuilder]:
"""Register an Orchestrator Function.
Parameters
Expand Down
Loading
Loading