Skip to content

JsonDataConverter.can_reconstruct omits dict/Mapping and tuple containers, skipping type-directed input reconstruction #344

Closed

Description

Problem Description

In durabletask/serialization.py, _coerce_generic() supports typed coercion for dict / Mapping and tuple containers:

https://github.com/microsoft/durabletask-python/blob/06bb27763eb893693f1bc300b86a86e01a5fa641/durabletask/serialization.py#L623-L645

    if origin in (dict, Mapping) and isinstance(value, dict):
        mapping = cast("dict[Any, Any]", value)
        val_type = args[1] if len(args) == 2 else None
        if val_type is None:
            return mapping
        return {k: _coerce_to_type(v, val_type, converter) for k, v in mapping.items()}
    if origin is tuple and isinstance(value, list):
        if not args:
            return tuple(cast(list[Any], value))
        if len(args) == 2 and args[1] is Ellipsis:
            return tuple(_coerce_to_type(item, args[0], converter) for item in cast(list[Any], value))
        arr = cast(list[Any], value)
        if len(arr) != len(args):
            raise TypeError(
                f"Could not coerce JSON array of length {len(arr)} to "
                f"tuple of length {len(args)}"
            )
        return tuple(_coerce_to_type(item, t, converter) for item, t in zip(arr, args))

And the module documentation explicitly states:

Type-directed reconstruction recurses through dataclass fields, list/Sequence, dict/Mapping values, tuple elements, and Optional / Union hints.

However, _can_reconstruct() only checks Union, list, Sequence, and set/frozenset:

https://github.com/microsoft/durabletask-python/blob/06bb27763eb893693f1bc300b86a86e01a5fa641/durabletask/serialization.py#L280-L295

def _can_reconstruct(converter: DataConverter, target_type: Any) -> bool:
    origin = typing.get_origin(target_type)
    if origin is not None:
        args = typing.get_args(target_type)
        if origin is typing.Union or origin is types.UnionType:
            return any(
                converter.can_reconstruct(a) for a in args if a is not type(None)
            )
        if origin in (list, Sequence):
            return any(converter.can_reconstruct(a) for a in args)
        if origin in (set, frozenset, AbstractSet, typing.AbstractSet):
            if not getattr(converter, "enable_generic_sets", False):
                return False
            if not args:
                return True
            return _is_supported_set_element(converter, args[0])
        return False

It completely omits dict, Mapping, and tuple.

Because converter.can_reconstruct(dict[str, MyDataclass]) and converter.can_reconstruct(tuple[MyDataclass, ...]) return False:
In durabletask/internal/type_discovery.py:

def activity_input_type(fn: Callable[..., Any], converter: DataConverter | None = None) -> type | None:
    ...
    return annotation if _resolve_converter(converter).can_reconstruct(annotation) else None

Inbound type discovery drops the annotation and resolves the target type to None. When the worker executes the activity or orchestrator:

input_type = type_discovery.activity_input_type(fn, self._data_converter) if encoded_input else None
activity_input = self._data_converter.deserialize(encoded_input, input_type)
activity_output = fn(ctx, activity_input)

Because input_type is None, the activity receives raw JSON dicts / lists instead of typed dataclass or model instances, resulting in runtime errors such as AttributeError: 'dict' object has no attribute '...'.


Minimal Reproduction

import json
from dataclasses import dataclass
from durabletask import worker
from durabletask.serialization import JsonDataConverter
from durabletask.internal import type_discovery

@dataclass
class User:
    name: str

def process_users_activity(ctx, users: dict[str, User]):
    for k, u in users.items():
        assert isinstance(u, User), f"Expected User for key {k}, got {type(u)}"
    return len(users)

converter = JsonDataConverter()

# 1. Type discovery returns None:
discovered_type = type_discovery.activity_input_type(process_users_activity, converter)
print("Discovered input type:", discovered_type)
assert discovered_type is None  # BUG: Expected dict[str, User]

# 2. Activity invocation fails:
reg = worker._Registry()
act_name = reg.add_activity(process_users_activity)
w = worker._ActivityExecutor(reg, worker.logging.getLogger(), converter)
payload = json.dumps({"u1": {"name": "Alice"}})

try:
    w.execute("orch-1", act_name, 1, payload)
except AssertionError as e:
    print("Activity execution failed:", e)

Output:

Discovered input type: None
Activity execution failed: Expected User for key u1, got <class 'dict'>

Expected Behavior

  1. JsonDataConverter.can_reconstruct(dict[K, V]) and can_reconstruct(Mapping[K, V]) should return True when V is reconstructable.
  2. JsonDataConverter.can_reconstruct(tuple[T, ...]) and can_reconstruct(tuple[T1, T2, ...]) should return True when the element type(s) are reconstructable.
  3. Inbound type discovery should preserve the annotation, allowing _coerce_generic() to reconstruct values into their designated dataclass/model types.

Impact

Users annotating activities or orchestrators with dict[str, MyModel], Mapping[str, MyModel], or tuple[MyModel, ...] receive uncoerced dictionaries and lists, bypassing type validation and failing at runtime during attribute access.


Suggested Direction

Update _can_reconstruct() in durabletask/serialization.py:

        if origin in (dict, Mapping):
            return len(args) == 2 and converter.can_reconstruct(args[1])
        if origin is tuple:
            if not args:
                return False
            if len(args) == 2 and args[1] is Ellipsis:
                return converter.can_reconstruct(args[0])
            return any(converter.can_reconstruct(a) for a in args)

cc andystaples Bernd Verst (@berndverst)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions