Add internal helper for batched modelzoo inference from in-memory arrays (inference runner) by deruyter92 · Pull Request #3222 · DeepLabCut/DeepLabCut

deruyter92 · 2026-02-26T17:15:53Z

Summary

Adds a new in-memory inference helper:
- create_superanimal_inference_runners(...)
- located in deeplabcut.pose_estimation_pytorch.modelzoo.inference
Exposes this helper from deeplabcut.pose_estimation_pytorch.modelzoo.__init__ for convenient imports.

Motivation

Currently it is fairly easy to run batched inference on our PyTorch models, but the modelzoo api is an exception that requires us to save images to disk first. This PR implements a modelzoo inference runner factory to easily run batched inference from in-memory image arrays. This can be used internally for faster inference. (Inspired by get_inference_runners).

This solves #3218

Example usage

from pathlib import Path
import numpy as np
from PIL import Image
from deeplabcut.pose_estimation_pytorch.modelzoo.inference import (
    create_superanimal_inference_runners,
)

img_paths = [
    "/path/to/images/frame_0000.png",
    "/path/to/images/frame_0001.png",
    "/path/to/images/frame_0002.png",
]
images = [np.asarray(Image.open(Path(p)).convert("RGB")) for p in img_paths]

pose_runner, det_runner, model_cfg = create_superanimal_inference_runners(
    superanimal_name="superanimal_quadruped",
    model_name="hrnet_w32",
    detector_name="fasterrcnn_resnet50_fpn_v2",
    max_individuals=10,
    batch_size=1,
    detector_batch_size=1,
)

det_preds = det_runner.inference(images)
pose_inputs = list(zip(images, det_preds))
pose_preds = pose_runner.inference(pose_inputs)

Introduce a modelzoo helper that resolves configs and checkpoints via build_weight_init, then constructs pose/detector runners for future in-memory batched APIs.

Copilot

Pull request overview

Adds a new Model Zoo helper to construct pose + detector inference runners for batched in-memory (array-based) SuperAnimal inference, and re-exports it for easier imports.

Changes:

Added create_superanimal_inference_runners(...) to build pose/detector InferenceRunners from an in-memory workflow.
Integrated config loading/updating + weight initialization (including optional custom checkpoints) into the runner factory.
Re-exported the helper from deeplabcut.pose_estimation_pytorch.modelzoo.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

File	Description
`deeplabcut/pose_estimation_pytorch/modelzoo/inference.py`	Adds the new runner-factory helper and required imports for config/weight init + device selection.
`deeplabcut/pose_estimation_pytorch/modelzoo/__init__.py`	Re-exports the new helper for convenient imports.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

deeplabcut/pose_estimation_pytorch/modelzoo/inference.py

C-Achard

Looks good so far, thanks !

deeplabcut/pose_estimation_pytorch/modelzoo/inference_helpers.py

Instead of several definitions of the persons class in the coco dataset, a single constant is placed in modelzoo.utils. This reduces code duplication improves maintainability and promotes centralized logic, rather than implying that the argument is intented to be tuned by the callers.

C-Achard

Really nice, I like the minimally invasive design.
If we're alright with making small tweaks under the hood, see comments, otherwise all good.

(Note : did not have the opportunity to test it yet)

deeplabcut/pose_estimation_pytorch/modelzoo/inference_helpers.py

… accordingly)

…orts)

MMathisLab

I did not run the PR, so maybe speed test this, but otherwise Lgtm!

deruyter92 · 2026-03-19T17:15:59Z

This PR basically exposes the inference runner that is used in superanimal image folder analysis, so you don't have the overhead of reading and writing images. As the backend inference runner is exactly the same (and you have to load the images at some point in your pipeline) the comparison is a bit unfair. But assuming that you already loaded the images in memory elsewhere in your script (the usecase of this PR), the benefits are as follows:

deruyter92 · 2026-03-19T17:16:51Z

Thanks for the review! I'll merge it once the tests are complete!

deruyter92 added 3 commits February 26, 2026 15:09

Add SuperAnimal runner factory for in-memory inference.

34b6192

Introduce a modelzoo helper that resolves configs and checkpoints via build_weight_init, then constructs pose/detector runners for future in-memory batched APIs.

Add docstring, typehints and formatting

ef78235

Move to pytorch inference (as it is specific to pytorch)

6ebb7a3

deruyter92 requested a review from Copilot February 26, 2026 17:16

Copilot started reviewing on behalf of deruyter92 February 26, 2026 17:16 View session

Copilot AI reviewed Feb 26, 2026

View reviewed changes

deruyter92 added 6 commits February 26, 2026 18:24

Fix circular import, move to separate file

0dee4a3

Fix bottom-up fucntionality: allow detector None

b453314

update docstring

3eb900e

Add specialcase for humanbody inference

f1ef0da

fix shallowcopy -> deepcopy

07cc8b1

add basic testing coverage for create_superanimal_inference_runners

5adffd6

C-Achard reviewed Feb 27, 2026

View reviewed changes

deeplabcut/pose_estimation_pytorch/modelzoo/inference_helpers.py Outdated Show resolved Hide resolved

deruyter92 added the enhancement New feature or request label Feb 28, 2026

deruyter92 marked this pull request as ready for review March 12, 2026 15:13

Merge branch 'main' into jaap/modelzoo_inference_api

56d09e0

deruyter92 requested review from AlexEMG and C-Achard March 12, 2026 15:14

C-Achard reviewed Mar 16, 2026

View reviewed changes

deruyter92 and others added 4 commits March 16, 2026 16:18

change autoresolve device: map default None -> "auto" (+ update tests…

04c4512

… accordingly)

torch modelzoo: add lazy export in __init__.py (to solve circular imp…

595085a

…orts)

omit num_bodyparts and num_unique_bodyparts (resolved downstream)

259315d

Merge branch 'main' into jaap/modelzoo_inference_api

513b886

deruyter92 requested a review from C-Achard March 16, 2026 15:29

C-Achard requested a review from MMathisLab March 19, 2026 15:16

MMathisLab approved these changes Mar 19, 2026

View reviewed changes

move update config before superanimal_inference_runners

1be89dc

deruyter92 merged commit 856dfac into main Mar 19, 2026
11 checks passed

deruyter92 deleted the jaap/modelzoo_inference_api branch March 19, 2026 19:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add internal helper for batched modelzoo inference from in-memory arrays (inference runner)#3222

Add internal helper for batched modelzoo inference from in-memory arrays (inference runner)#3222
deruyter92 merged 16 commits intomainfrom
jaap/modelzoo_inference_api

deruyter92 commented Feb 26, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

C-Achard left a comment

Uh oh!

Uh oh!

C-Achard left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MMathisLab left a comment

Uh oh!

deruyter92 commented Mar 19, 2026 •

edited

Loading

Uh oh!

deruyter92 commented Mar 19, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

deruyter92 commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Motivation

Example usage

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

C-Achard left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

C-Achard left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MMathisLab left a comment

Choose a reason for hiding this comment

Uh oh!

deruyter92 commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

deruyter92 commented Mar 19, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

deruyter92 commented Feb 26, 2026 •

edited

Loading

deruyter92 commented Mar 19, 2026 •

edited

Loading