Skip to content

TypeError: CondPreNet.forward() missing 1 required positional argument: 'cond_kpts' #3001

Description

@HLH2000

Is there an existing issue for this?

  • I have searched the existing issues

Operating System

Win11

DeepLabCut version

3.0.0rc8

What engine are you using?

pytorch

DeepLabCut mode

multi animal

Device type

RTX 3080

Bug description 🐛

After running the video analysis process once without specifying an output destination, I attempted to run it again with destfolder=dest_folder. However, an error occurred: TypeError: CondPreNet.forward() missing 1 required positional argument: 'cond_kpts'.

Steps To Reproduce

Initially, I ran this code, and no error messages appeared.

videofile_path=str(r"C:\Users\HLH\Desktop\video")
deeplabcut.analyze_videos(
    config,
    [videofile_path],
    shuffle=CTD_SHUFFLE,
    ctd_tracking=True,
    save_as_csv=True 
)
deeplabcut.create_labeled_video(
    config,
    [videofile_path],
    shuffle=CTD_SHUFFLE,
    track_method="ctd",
    color_by="individual",
)

Then, I tried adjusting the parameters, and an error occurred.

videofile_path=str(r"C:\Users\HLH\Desktop\video")
dest_folder = str(Path(videofile_path).parent / "custom-ctd-tracking")
deeplabcut.analyze_videos(
    config,
    [videofile_path],
    shuffle=CTD_SHUFFLE,
    destfolder=dest_folder,
    ctd_tracking=dict(
        bu_on_lost_idv=True,
        bu_max_frequency=50,
        bu_min_frequency=1,
        threshold_bu_add=0.5,
        threshold_ctd=0.01,
        threshold_nms=0.8,
    ),
)
deeplabcut.create_labeled_video(
    config,
    [videofile_path],
    shuffle=CTD_SHUFFLE,
    destfolder=dest_folder,
    track_method="ctd",
    color_by="individual",
)

Relevant log output

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[7], line 4
      1 videofile_path=str(r"C:\Users\HLH\Desktop\video")
      2 dest_folder = str(Path(videofile_path).parent / "custom-ctd-tracking02")
----> 4 deeplabcut.analyze_videos(
      5     config,
      6     [videofile_path],
      7     shuffle=CTD_SHUFFLE,
      8     destfolder=dest_folder,
      9     ctd_tracking=dict(
     10         bu_on_lost_idv=True,
     11         bu_max_frequency=100,
     12         bu_min_frequency=20,
     13         threshold_bu_add=0.5,
     14         threshold_ctd=0.01,
     15         threshold_nms=0.8,
     16     ),
     17 )
     18 deeplabcut.create_labeled_video(
     19     config,
     20     [videofile_path],
   (...)
     24     color_by="individual",
     25 )

File ~\AppData\Local\anaconda3\envs\dlc3\lib\site-packages\deeplabcut\compat.py:954, in analyze_videos(config, videos, videotype, shuffle, trainingsetindex, gputouse, save_as_csv, in_random_order, destfolder, batchsize, cropping, TFGPUinference, dynamic, modelprefix, robust_nframes, allow_growth, use_shelve, auto_track, n_tracks, animal_names, calibrate, identity_only, use_openvino, engine, **torch_kwargs)
    951         else:
    952             torch_kwargs["batch_size"] = batchsize
--> 954     return analyze_videos(
    955         config,
    956         videos=videos,
    957         videotype=videotype,
    958         shuffle=shuffle,
    959         trainingsetindex=trainingsetindex,
    960         save_as_csv=save_as_csv,
    961         in_random_order=in_random_order,
    962         destfolder=destfolder,
    963         dynamic=dynamic,
    964         modelprefix=modelprefix,
    965         use_shelve=use_shelve,
    966         robust_nframes=robust_nframes,
    967         auto_track=auto_track,
    968         n_tracks=n_tracks,
    969         animal_names=animal_names,
    970         calibrate=calibrate,
    971         identity_only=identity_only,
    972         overwrite=False,
    973         cropping=cropping,
    974         **torch_kwargs,
    975     )
    977 raise NotImplementedError(f"This function is not implemented for {engine}")

File ~\AppData\Local\anaconda3\envs\dlc3\lib\site-packages\deeplabcut\pose_estimation_pytorch\apis\videos.py:545, in analyze_videos(config, videos, videotype, shuffle, trainingsetindex, save_as_csv, in_random_order, snapshot_index, detector_snapshot_index, device, destfolder, batch_size, detector_batch_size, dynamic, ctd_conditions, ctd_tracking, top_down_dynamic, modelprefix, use_shelve, robust_nframes, transform, auto_track, n_tracks, animal_names, calibrate, identity_only, overwrite, cropping, save_as_df)
    543 else:
    544     runtime = [time.time()]
--> 545     predictions = video_inference(
    546         video=video_iterator,
    547         pose_runner=pose_runner,
    548         detector_runner=detector_runner,
    549         shelf_writer=shelf_writer,
    550         robust_nframes=robust_nframes,
    551     )
    552     runtime.append(time.time())
    553     metadata = _generate_metadata(
    554         cfg=loader.project_cfg,
    555         pytorch_config=loader.model_cfg,
   (...)
    562         robust_nframes=robust_nframes,
    563     )

File ~\AppData\Local\anaconda3\envs\dlc3\lib\site-packages\deeplabcut\pose_estimation_pytorch\apis\videos.py:204, in video_inference(video, pose_runner, detector_runner, cropping, shelf_writer, robust_nframes)
    201 if shelf_writer is not None:
    202     shelf_writer.open()
--> 204 predictions = pose_runner.inference(images=tqdm(video), shelf_writer=shelf_writer)
    205 if shelf_writer is not None:
    206     shelf_writer.close()

File ~\AppData\Local\anaconda3\envs\dlc3\lib\site-packages\torch\utils\_contextlib.py:116, in context_decorator.<locals>.decorate_context(*args, **kwargs)
    113 @functools.wraps(func)
    114 def decorate_context(*args, **kwargs):
    115     with ctx_factory():
--> 116         return func(*args, **kwargs)

File ~\AppData\Local\anaconda3\envs\dlc3\lib\site-packages\deeplabcut\pose_estimation_pytorch\runners\inference.py:394, in CTDInferenceRunner.inference(self, images, shelf_writer)
    374 """Run CTD model inference on the given dataset
    375 
    376 Args:
   (...)
    391     ]
    392 """
    393 if self.tracking:
--> 394     return self._ctd_tracking_inference(images, shelf_writer)
    396 results = []
    397 for data in images:

File ~\AppData\Local\anaconda3\envs\dlc3\lib\site-packages\deeplabcut\pose_estimation_pytorch\runners\inference.py:491, in CTDInferenceRunner._ctd_tracking_inference(self, images, shelf_writer)
    489 inputs, context = self._prepare_ctd_inputs(data)
    490 model_kwargs = context.pop("model_kwargs", {})
--> 491 predictions = self.predict(inputs, **model_kwargs)
    492 if self.postprocessor is not None:
    493     # Pop the "cond_kpts" from the context so there's no re-scoring
    494     # This is required when tracking with CTD, otherwise scores go to 0
    495     if self._prev_pose is not None:

File ~\AppData\Local\anaconda3\envs\dlc3\lib\site-packages\deeplabcut\pose_estimation_pytorch\runners\inference.py:427, in CTDInferenceRunner.predict(self, inputs, **kwargs)
    410 def predict(
    411     self, inputs: torch.Tensor, **kwargs
    412 ) -> list[dict[str, dict[str, np.ndarray]]]:
    413     """Makes predictions from a model input and output
    414 
    415     Args:
   (...)
    425         ]
    426     """
--> 427     outputs = self.model(inputs.to(self.device), **kwargs)
    428     raw_predictions = self.model.get_predictions(outputs)
    429     predictions = [
    430         {
    431             head: {
   (...)
    437         for b in range(len(inputs))
    438     ]

File ~\AppData\Local\anaconda3\envs\dlc3\lib\site-packages\torch\nn\modules\module.py:1751, in Module._wrapped_call_impl(self, *args, **kwargs)
   1749     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1750 else:
-> 1751     return self._call_impl(*args, **kwargs)

File ~\AppData\Local\anaconda3\envs\dlc3\lib\site-packages\torch\nn\modules\module.py:1762, in Module._call_impl(self, *args, **kwargs)
   1757 # If we don't have any hooks, we want to skip the rest of the logic in
   1758 # this function, and just call forward.
   1759 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1760         or _global_backward_pre_hooks or _global_backward_hooks
   1761         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1762     return forward_call(*args, **kwargs)
   1764 result = None
   1765 called_always_called_hooks = set()

File ~\AppData\Local\anaconda3\envs\dlc3\lib\site-packages\deeplabcut\pose_estimation_pytorch\models\model.py:78, in PoseModel.forward(self, x, **backbone_kwargs)
     76 if x.dim() == 3:
     77     x = x[None, :]
---> 78 features = self.backbone(x, **backbone_kwargs)
     79 if self.neck:
     80     features = self.neck(features)

File ~\AppData\Local\anaconda3\envs\dlc3\lib\site-packages\torch\nn\modules\module.py:1751, in Module._wrapped_call_impl(self, *args, **kwargs)
   1749     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1750 else:
-> 1751     return self._call_impl(*args, **kwargs)

File ~\AppData\Local\anaconda3\envs\dlc3\lib\site-packages\torch\nn\modules\module.py:1762, in Module._call_impl(self, *args, **kwargs)
   1757 # If we don't have any hooks, we want to skip the rest of the logic in
   1758 # this function, and just call forward.
   1759 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1760         or _global_backward_pre_hooks or _global_backward_hooks
   1761         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1762     return forward_call(*args, **kwargs)
   1764 result = None
   1765 called_always_called_hooks = set()

TypeError: CondPreNet.forward() missing 1 required positional argument: 'cond_kpts'

Anything else?

I believe this issue is not caused by ctd_tracking=dict(.....). Instead, if there are any analyzed results in the folder (videofile_path) where the raw videos are located, the error appears.

I have another question. I compared the results between BUCTD and the traditional method (ResNet50 + auto-tracking). The BUCTD model showed more instability in body part tracking. Is there any way to improve the stability or use the auto-tracking results as a reference for the BUCTD model?

Code of Conduct

Metadata

Metadata

Assignees

Labels

CTDContidional Top-Down

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions