Skip to content

Latest commit

 

History

History
154 lines (124 loc) · 6.78 KB

development.md

File metadata and controls

154 lines (124 loc) · 6.78 KB

Development

The following is more information about how to develop this project.

Validation

Use Nox to run the representative local correctness checks:

python -m pip install --upgrade "pip>=25.1"
python -m pip install --group nox
nox -s ci

Development dependencies are defined once in the root pyproject.toml under [dependency-groups] (PEP 735). Installing groups directly with pip requires pip 25.1 or later. Nox uses its built-in group resolver to install each session's tools without upgrading pip inside the session. The official build uses the same build, lint, and test groups. Add test-only dependencies to test so local and official test environments stay in sync. Keep runtime dependencies and public extras in each package's [project] metadata, not in tooling groups: the official build resolves them through built wheels to catch missing declarations.

For development without Nox, install all tools with --group dev and install the local packages separately. Use Python 3.13 or later for Azure Functions:

python -m pip install --group dev \
  -e ".[azure-blob-payloads,opentelemetry]" \
  -e ./durabletask-azuremanaged -e ./azure-functions-durable
python -m pip install --group dev `
  -e ".[azure-blob-payloads,opentelemetry]" `
  -e .\durabletask-azuremanaged -e .\azure-functions-durable

Sample and Function app requirements.txt files remain deployment inputs, not the source of repository development dependencies.

The ci session runs linting, strict type checks, core SDK and Azure Managed tests on Python 3.11, Azure Functions unit tests on Python 3.13, and Azure Functions end-to-end tests on Python 3.13. It intentionally runs one representative version rather than the complete CI matrix, keeping routine local validation fast. To use another supported version (3.11--3.15) for the core SDK and Azure Managed tests, pass it after --:

nox -s ci -- 3.15

Run the versioned test sessions directly when a change needs complete matrix coverage, for example nox -s core_tests or nox -s azuremanaged_tests. Azure Functions unit tests cover Python 3.13--3.15; use nox -s functions_unit-3.15 to target Python 3.15 specifically. Nox fails rather than silently skipping a session when its required Python interpreter is unavailable.

Modernization and deprecation checks

nox -s lint runs Flake8 and Ruff's UP modernization rules; autopep8 remains the formatter. Ruff uses Python 3.11 for core/Managed code and Python 3.13 for Functions code, including their respective tests and examples. Generated protobuf files are excluded, including when passed explicitly. For a focused check, run from the repository root:

python -m ruff check durabletask/client.py

The temporary typing and UTC rule exclusions in pyproject.toml are tracked in #322 and #323. Behavior-sensitive generator and enum suggestions have file-specific exceptions tracked in #324. Excluded rules do not reject new occurrences until their cleanup is complete; they are not recommendations for new code.

The root pytest configuration treats DeprecationWarning and PendingDeprecationWarning as errors and explicitly uses function-scoped event loops for async fixtures. Nox uses Python 3.13 as the representative strict deprecation gate for all three packages. On other supported versions, Nox reports these warnings without failing, preserving compatibility coverage without requiring APIs unavailable at the package's minimum version. Run the strict sessions with:

nox -s core_tests-3.13 azuremanaged_tests-3.13 functions_unit-3.13

Direct pytest runs from the repository root are strict on every interpreter. Tests intentionally exercising deprecated APIs should use pytest.warns. Any known third-party warning exception must be narrowly scoped by category, message, and originating module where practical, and include a reason. Two upstream Functions loose-codec warnings remain visible but non-fatal while legacy payload support is required; their removal is tracked in #325. Warnings raised in separately launched Functions workers are not governed by the parent pytest process's filters.

Services and focused validation

Nox starts Azurite automatically for the core and Azure Functions tests. The Azure Managed tests start a disposable DTS emulator Docker container. Start Docker before running that session. Azure Functions end-to-end tests also require the Azure Functions Core Tools (func) on PATH. Azure Managed Nox runs use a disposable DTS emulator with an automatically assigned port, and Functions E2E uses a unique Durable Functions hub. Both can run alongside other local validation sessions.

CodeQL remains a GitHub-hosted security scan; it requires the CodeQL CLI and its query packs, so it is not included in the local ci session.

For iterative validation, use -R to reuse the session virtual environment. The local packages are editable in test sessions, so source changes are picked up without reinstalling. Pass file or directory paths after -- to lint and type-check sessions. Pytest-based sessions also accept test node IDs and pytest selectors such as -k. Nox forwards these arguments unchanged to the underlying tool, and multiple paths or selectors are supported.

For a focused change, run just the relevant session:

nox -R -s lint -- durabletask/client.py tests/durabletask/test_client.py
nox -R -s typecheck_core -- durabletask/client.py durabletask/extensions/history_export
nox -R -s typecheck_functions -- azure-functions-durable/azure
nox -R -s core_tests-3.11 -- tests/durabletask/test_client.py::test_get_grpc_channel_insecure
nox -R -s azuremanaged_tests-3.11 -- tests/durabletask-azuremanaged/test_dts_orchestration_e2e.py
nox -R -s functions_unit-3.13 -- tests/azure-functions-durable/test_client_compat.py
nox -R -s functions_e2e -- -k "dtask_client"

Note

The Azure Managed session still starts a fresh isolated DTS emulator for every run, and Functions E2E starts a fresh Functions host. -R skips environment provisioning, not these required runtime services.

Generating protobufs

These commands require make. If you're using Windows, you can install make using Chocolatey or use WSL.

python -m pip install --upgrade "pip>=25.1"
python -m pip install --group protobuf
make gen-proto

This will download the orchestrator_service.proto from the microsoft/durabletask-protobuf repo and compile it using grpcio-tools. The version of the source proto file that was downloaded can be found in the file durabletask/internal/PROTO_SOURCE_COMMIT_HASH.