Skip to content

[Bug] DeepLabCut 3.0 PyTorch with AMD GPU (DirectML) fails on ConvTranspose2d in inference_mode — Workaround: use no_grad #3289

Description

@IrfaAPrayogi

Is there an existing issue for this?

  • I have searched the existing issues

Operating System

Window 11

DeepLabCut version

3.0.0rc14 (PyTorch engine)

What engine are you using?

pytorch

DeepLabCut mode

single animal

Device type

AMD Radeon RX 5700 XT

Bug description 🐛

When running analyze_videos using an AMD GPU via the DirectML backend, the process fails with the following error:
RuntimeError: Cannot set version_counter for inference tensor
This occurs during the ConvTranspose2d operation in the model's head.

Steps To Reproduce

The issue is caused by the @torch.inference_mode() decorator in pose_estimation_pytorch/runners/inference.py. Tensors created in inference_mode are immutable and have restricted version counting, which is currently incompatible with the DirectML backend for specific convolutional kernels like ConvTranspose2d.

Relevant log output

Final Verified Workaround
Replacing `@torch.inference_mode()` with `@torch.no_grad()` in the following file/lines fixes the issue and allows the AMD GPU to be utilized correctly via DirectML:
**File:** `deeplabcut/pose_estimation_pytorch/runners/inference.py`
**Lines:** 267 and 682
By switching to `no_grad`, the tensors are compatible with the DirectML execution path, enabling full hardware acceleration for AMD users.

Anything else?

No response

Code of Conduct

Metadata

Metadata

Assignees

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions