Skip to content

Documentation: napari-deeplabcut CoTracker integration - #3318

Merged
MMathisLab merged 13 commits into
cy/docs-audit-2026-toc-updatefrom
cy/docs-update-napari-tracking
May 21, 2026
Merged

Documentation: napari-deeplabcut CoTracker integration#3318
MMathisLab merged 13 commits into
cy/docs-audit-2026-toc-updatefrom
cy/docs-update-napari-tracking

Conversation

@C-Achard

@C-Achard C-Achard commented May 8, 2026

Copy link
Copy Markdown
Collaborator

Adds documentation for the new automated point tracking feature in the napari plugin.
The main focus is on guiding users through the new tracking workflow, requirements, and troubleshooting, as well as providing proper attribution and empirical limitations for the tracking models.

New documentation for automated point tracking:

  • Added a new page, docs/gui/napari/tracking/basic_usage.md, with detailed instructions and workflow for using the tracking feature in the napari plugin, including requirements, UI walkthrough, keyboard shortcuts, result interpretation, and troubleshooting.
  • Updated the table of contents (_toc.yml) to include the new tracking documentation under the napari section.

Required

C-Achard added 4 commits May 8, 2026 14:08
Shorten top-level headings in the napari documentation by removing the redundant "napari-DLC -" prefix for consistency and brevity. Updated docs/gui/napari/basic_usage.md and docs/gui/napari/advanced_usage.md to use simpler headings.
Add a new user guide page for the Tracking Controls widget (docs/gui/napari/tracking/basic_usage.md) describing UI, workflow, requirements, tracking actions, refinement/merge tools, model attribution (CoTracker3), troubleshooting, and limitations. Also add the accompanying controls image (docs/images/napari/tracking/controls.png). These additions document how to run point tracking, refine results, and merge tracked points back into DeepLabCut projects.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new documentation page describing the napari-deeplabcut automated point tracking workflow (CoTracker integration) and wires it into the docs navigation, plus a small cleanup to existing napari page headings.

Changes:

  • Added a new napari tracking user guide page covering requirements, UI walkthrough, workflow, troubleshooting, and model attribution/limitations.
  • Updated _toc.yml to include the new tracking page under the napari section.
  • Simplified the titles of existing napari “basic” and “advanced” usage pages.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 5 comments.

File Description
docs/gui/napari/tracking/basic_usage.md New end-to-end tracking workflow documentation and model attribution section.
docs/gui/napari/basic_usage.md Heading/title cleanup for the basic usage page.
docs/gui/napari/advanced_usage.md Heading/title cleanup for the advanced usage page.
_toc.yml Adds the tracking docs page to the napari documentation TOC.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/gui/napari/tracking/basic_usage.md
Comment thread docs/gui/napari/tracking/basic_usage.md Outdated
Comment thread docs/gui/napari/tracking/basic_usage.md Outdated
Comment thread docs/gui/napari/tracking/basic_usage.md Outdated
Comment thread docs/gui/napari/tracking/basic_usage.md Outdated
C-Achard added 2 commits May 8, 2026 14:40
Update docs/gui/napari/tracking/basic_usage.md to improve clarity and fix typos: rename "Keypoint Controls layer" to "Keypoint Controls dock widget", correct "Emprical" to "Empirical", and small whitespace/wording tweaks. Restructure the Limitations/Future directions area by adding "Important considerations" and "Future features" headings, move and reword the CoTracker3 note into Future features, consolidate and bullet manual curation and training-set imbalance guidance, and remove duplicate lines. These changes improve readability and provide clearer guidance for users.
@C-Achard

C-Achard commented May 8, 2026

Copy link
Copy Markdown
Collaborator Author

Do we want more images ?

C-Achard added 2 commits May 8, 2026 14:57
Clean up docs/gui/napari/tracking/basic_usage.md: add missing blank line in a tip, normalize bullet indentation and spacing, fix inconsistent numbered lists (renumber steps and convert some items to sub-bullets), and improve clarity in workflow instructions. Also add a brief consideration that manual annotation can sometimes be faster than heavy tracking corrections and note preference for continuous frames. These are purely documentation/formatting edits to improve readability and usability.
Revise and clarify the Tracking Controls documentation: wrap third-party model attribution in a note, add a brief reminder that tracking accelerates but does not replace manual review, and split Requirements into “In napari” and “In your Python environment”. Improve UI documentation with a control/description table, a figure placeholder, and a note about available models. Reword and tighten many instructions (frame/reference wording, tracking range, actions, keyboard shortcuts), clarify that tracking runs in a background worker, and explain that tracking-result layers are intermediate and must be merged/saved to update DLC project files. Add a prominent warning that there is currently no undo for deletion/merge operations. Minor formatting and grammar fixes throughout (CoTracker attribution links, troubleshooting bullets, phrasing) to improve readability and consistency.
@C-Achard C-Achard added the napari-dlc Related to napari-deeplabcut (labeling workflow, crop, frame extraction...) label May 8, 2026
@C-Achard
C-Achard requested a review from deruyter92 May 8, 2026 13:20
C-Achard added a commit to DeepLabCut/napari-deeplabcut that referenced this pull request May 18, 2026
@C-Achard
C-Achard marked this pull request as ready for review May 18, 2026 11:06
C-Achard added a commit to DeepLabCut/napari-deeplabcut that referenced this pull request May 18, 2026
* Add experimental tracking feature (UI, worker, core)

Introduce an experimental point-tracking subsystem: adds a TrackingControls UI, a background TrackingWorker, and core data/models for registering and running trackers. Implements concrete modules under tracking/core (data and abstract TrackingModel with registry), a Qt widget under tracking/_widgets.py, and a worker implementation under tracking/_worker.py. Includes tests (fixtures, widget and worker tests) and a README documenting usage and keybindings; tests register a DummyTracker for unit testing. Adds "torch" to pyproject.toml and updates tox.ini to support the new tracking tests.

Co-Authored-By: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com>

* Restore plugin tracking manifest

Co-Authored-By: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com>

* Make tracking shortcuts configurable & conditional

Introduce configurable tracking shortcuts and only enable them when the tracking widget is visible. Add TRACKING_SHORTCUTS_ENABLED (env NAPARI_DLC_TRACKING_SHORTCUTS_ENABLED, default enabled) to settings, centralize tracking key definitions in config/keybinds.py, and have iter_shortcuts yield tracking shortcuts only when enabled. Update TrackingControls to import those key configs, set an objectName/property for dock detection, and guard bound key callbacks so they only act when the tracking dock is open/visible. Enhance Shortcuts dialog to recognize the new "tracking-points-layer" scope and to hide/disable shortcuts when the tracking widget isn't open. Also update imports (KeypointStore path) and remove duplicate dataclass/keybind definitions.

* Fix shortcut tests

* Improve tracking feature pipeline and result layers

Introduce an identity-preserving tracking data pipeline and create separate tracking result layers. data.py: add tracking metadata constants, convert TrackingWorkerData/Output to use DataFrame for per-query features, and add helpers (coerce_features_df, add_query_identity_columns, expand_query_features_over_time, build_tracking_result_metadata, is_tracking_result_layer) to preserve query identity and expand features over time. models.py: update Cotracker3 to avoid mutating inputs, correctly handle x/y ordering, visibility shapes, time-reversal for backward tracking, and produce flattened keypoints plus expanded per-point feature rows. _widgets.py: add UI helpers to seed query points/features, build a non-destructive tracking result Points layer (with metadata and visual tweaks), wire those into the tracking flow, improve error handling, and select the created layer in the viewer. Overall these changes ensure semantic identity of query points is preserved through inference and that results are stored in a dedicated, annotated layer.

* Fix tracking tests

* Fix incorrect dataclass use

* Improve tracking tests: seed frames and assertions

Add helpers to place all test keypoints on a specific frame and to drive the widget reference frame (_put_all_points_on_frame, _set_current_tracking_frame). Update tests (test_backward_track, test_bothway_track) to use these helpers instead of directly mutating viewer.dims, and add assertions verifying the tracking request payload: correct reference_frame_index, sliced video length, keypoints rebased to local frame 0, original keypoint feature columns (id/name) preserved, and new tracking identity columns (tracking_query_index, tracking_query_frame) populated. Also ensure both-way tracking uses the same seed frame and that forward==reference only triggers backward tracking.

* Remove duplicate worker

* Use queued signals and refine TrackingWorker signals

Switches several signal/slot connections to explicit Qt.QueuedConnection to ensure safe cross-thread delivery and replaces tuple/object-typed signals with primitive signatures. In TrackingControls: trackingRequested now uses Signal(object); worker connections use queued delivery; added _on_worker_started/_on_worker_finished/_on_worker_progress slots, a _request_worker_stop wrapper, and a _debug_thread helper for logging. In TrackingWorker: progress is now Signal(int,int), trackingFinished is Signal(object), track is annotated with Slot(object), and progress.emit now emits two ints; added thread debug logging. These changes improve thread-safety, clarity of signal payloads, and progress reporting.

* Refactor TrackingWorker stop flow and logging

Update tracking worker and UI to improve stop handling, logging, and error reporting.

- TrackingWorker: replace boolean stop flag with threading.Event, add request_stop(), use self.thread for logging, emit trackingStarted/trackingStopped/trackingFinished and finished in appropriate places, avoid QCoreApplication.processEvents, and guard torch.cuda.empty_cache. Ensure early return when stop requested.
- Tests: adapt progress signal handling to accept (current, total) args.
- Widgets: use self.thread attribute in debug logs, derive reference frame index from returned features when available, catch ValueError from seed point queries and show a napari warning instead of crashing, plus small formatting fixes.

These changes make worker shutdown safer across threads, improve diagnostics, and prevent main-thread operations from being performed in the worker.

* Use CoTracker offline model and fix visibility

Switch the CoTracker backend to the offline model and refactor run() to perform whole-clip inference. Inputs.video is converted and permuted to (1, T, C, H, W), queries are batched to (1, K, 3), inference is executed under torch.inference_mode, progress callbacks updated, and a RuntimeError is raised if no predictions are returned. Also add early-stop handling (return None). Additionally, fix visibility array handling in expand_query_features_over_time by squeezing a leading singleton channel dimension when vis.ndim == 3 and vis.shape[0] == 1.

* Adjust tracking UI visuals and threading bugfix

Tweak tracking UI appearance and fix a threading/logging bug.

- tracking/_widgets.py: add TODO about slider sync; change tracking layer visuals to use symbol="cross", opacity=0.85, and border_color="green" to better distinguish results.
- tracking/ui/worker.py: fix logging call by using self.thread instead of self.thread() to avoid a mistaken call in the thread information.

* Update src/napari_deeplabcut/tracking/ui/worker.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/napari_deeplabcut/tracking/core/models.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/napari_deeplabcut/_tests/config/test_keybinds.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Improve tracking widgets and lazy torch import

Use a lifecycle manager in TrackingControls and tighten viewer typing; update test helper to locate the TrackingControls dock by isinstance and add TYPE_CHECKING napari types. Fix a TYPE_CHECKING import in tracking models (correct TrackingModel import). Replace top-level torch import in the tracking worker with a lazy import and a clear ImportError message advising how to install the tracking extras, and improve CUDA cache cleanup logging. These changes reduce heavy top-level imports, improve typing correctness, and make missing-dependency errors clearer.

* Run pre-commit

* Tracking UI: Merge tracking results (#190)

* Add experimental tracking feature (UI, worker, core)

Introduce an experimental point-tracking subsystem: adds a TrackingControls UI, a background TrackingWorker, and core data/models for registering and running trackers. Implements concrete modules under tracking/core (data and abstract TrackingModel with registry), a Qt widget under tracking/_widgets.py, and a worker implementation under tracking/_worker.py. Includes tests (fixtures, widget and worker tests) and a README documenting usage and keybindings; tests register a DummyTracker for unit testing. Adds "torch" to pyproject.toml and updates tox.ini to support the new tracking tests.

Co-Authored-By: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com>

* Add tracking merge logic for Points layers

Introduce src/napari_deeplabcut/tracking/core/merge.py implementing merge preview and apply workflows for merging tracking-result Points into DLC Points layers. Adds data models (TrackingMergePreview, LayerFingerprint, TrackingMergeConflictEntry) and core APIs preview_tracking_merge and apply_tracking_merge with staleness checks via fingerprints, conflict detection, and a FILL_MISSING policy. Includes robust normalization/validation helpers for Napari Points data/features, coordinate-tolerance comparisons, duplicate-slot detection, and safe feature merging that preserves target schema.

* Support tracking-result Points layers

Add lifecycle manager support for DLC tracking-result Points layers: import the tracking metadata helpers and expose methods to read tracking metadata, classify tracking-result layers, extract source layer names, detect mergeable DLC Points layers, iterate tracking/mergeable layers, and suggest a default merge target. These methods centralize viewer/session-facing semantics for tracking results and provide merge-selection logic (managed vs live, name matching, active layer preference). Also rename the helper in tracking core from is_tracking_result_layer to is_tracking_result_points_layer and update imports accordingly.

* Add tracking merge UI and workflow

Introduce a new tracking merge UI and workflow to merge tracked points into DLC Points layers. Adds a new module tracking/ui/merger.py implementing TrackingMergeWorkflow, TrackingMergeDialog, and TrackingMergeConflictsDialog with preview, conflict reporting, and application logic (uses preview_tracking_merge and apply_tracking_merge). Integrates a "Merge tracked points…" button into TrackingControls (_widgets.py), wiring it to open the workflow (with a hinted active source), and adds the button to the controls layout. The workflow validates candidates, shows transactional dialogs, applies the merge to the target layer, updates the viewer selection/status, and marks layer presentation changes with error handling and user-facing warnings.

* Add tests for tracking merge behaviors

Add comprehensive tests for tracking merge utilities: fingerprint_points_layer, preview_tracking_merge, and apply_tracking_merge. Tests cover valid/invalid previews (same layer, duplicate semantic slots), classification of identical/conflict/appendable/invalid rows, preservation of target schema and prevention of source-only column leakage, handling of no-op merges (returns copies), detection of stale previews when layer name or feature columns change, use of id in semantic identity, and coordinate-tolerance effects on identical vs conflict. Also add a small header comment to the tracking conftest.

* Restore changes from parent branch

* Include preview reason in invalid merge error

Improve diagnostics when applying an invalid tracking merge preview by appending preview.invalid_reason to the raised ValueError (defaults to 'Unknown reason.'). This change makes the failure message more informative without altering control flow or behavior.

* Escape layer names in merge UI, simplify logs

HTML-escape source/target layer display names shown in the tracking merge dialogs to avoid HTML injection and rendering issues (adds html.escape import). Also remove explicit exc_info argument passed to logger.exception calls, relying on logger.exception to include exception info by default.

* Validate coordinate columns before merge

Add a pre-merge validation that ensures the source (append_df) contains all coordinate columns required by the target layer. If any coord_* columns are missing, the code now raises a ValueError with a clear message listing the missing required columns and the coord_* columns present in the source, and suggests refreshing the preview or merging into a compatible target. This prevents silent dimensionality mismatches and potential data corruption during tracking merges.

* Add review-only mode to merge conflicts dialog

Introduce a review_only mode to TrackingMergeConflictsDialog so it can be used both as an informational "Review conflicts" viewer and as a confirmation dialog before applying a partial merge. Adds a review_only parameter, adjusts window titles and summary text, and changes the button row (Close for review mode; Cancel/Merge for confirm mode). Exposes a new static review(...) helper and calls it from the preview path. Also simplifies label text construction by using preview.source_layer_name/target_layer_name directly.

* Make merge button more distinct

* Handle tracking-result points in trajectory plot

Integrate the LayerLifecycleManager with the trajectory canvas and improve selection of plottable Points layers. Add manager methods (is_plottable_traj_layer, iter_plottable_traj_layers, suggest_plottable_traj_layer) to determine and suggest eligible layers in viewer order, prioritizing active and managed layers. Have TrajectoryMatplotlibCanvas use get_or_create_layer_manager, refresh on layer insert/remove/selection events, and ask the manager for the plot layer. Add logic to merge tracking-result layer.features and properties into a properties mapping so tracking outputs can be plotted, and use that mapping when building the DataFrame for plotting. Add defensive checks and debug logs to avoid plotting incomplete or incompatible Points layers.

* Standardize logger names to module __name__

Replace hard-coded logger strings with logging.getLogger(__name__) in several modules (widgets, layer lifecycle manager and registry) and update tests and conftest to use the underscore package name 'napari_deeplabcut' instead of the hyphen variant. This aligns logger names with module __name__ and keeps logging configuration and test expectations consistent; no functional behavior changes intended.

* Add future-delete refine, utils; refactor merge

Introduce tracking refine and utility modules and wire a new UI action to delete tracked points in future frames.

- Add tracking/core/refine.py: preview_delete_tracking_points_in_future and apply_delete_tracking_points_in_future plus TrackingFutureDeletePreview dataclass to support previewing and applying bulk deletion of matching tracked points in future frames.
- Add tracking/core/utils.py: utilities for robust Points layer handling (extract_layer_data_and_features, coord_columns_for_data, pick_semantic_series, coerce_frame_series, normalize_slot_id, slot key helpers, normalize_points_layer_for_tracking) and helpers for generating stable tracked-layer names (make_tracking_iteration_name and related helpers).
- Update widgets (_widgets.py): add a "Delete selected points in future frames" button, preview/apply flow with confirmations and notifications, improved error messages, use make_tracking_iteration_name for new tracked layers, and minor UI/log cleanup.
- Refactor tracking/core/merge.py to reuse the new utils (coord/normalize/extract/format helpers), remove duplicated helper implementations, and adapt preview/apply logic to new utility APIs.

These changes centralize Points data/feature normalization and naming logic, add a user-facing refine action for tracked predictions, and reduce duplication across merge/refine code paths.

* Refactor tracking name generation & add typing

Add TYPE_CHECKING imports and napari type hints; update spawn.py viewer helpers to use forward-referenced napari.Viewer types (non-functional). Refactor tracking utils to accept an explicit LayerLifecycleManager instead of relying on self, move get_or_create_layer_manager import into the module, and derive a family regex via _tracking_version_family_pattern so tracked-layer versions are incremented per (source, tracker) family regardless of reference frame. Small cleanup and formatting adjustments.

* Add deferred keypoint combo selection with QTimer

Import QTimer and add _select_keypoint_combo_layer to TrackingControls. The new helper resets the keypoint combo choices and schedules selecting a newly added Points layer on the next event-loop tick to avoid race conditions during layer insertion. Includes robust try/except logging around choice reset and selection, and leaves a FIXME about the unowned QTimer usage.

* Update _widgets.py

* Update widget_factory.py

* Run pre-commit

* Add overwrite-existing merge policy

Introduce a new TrackingMergePolicy.OVERWRITE_EXISTING and extend the merge preview/apply workflow to support overwriting existing target points. preview_tracking_merge now classifies mismatches as either conflicts (FILL_MISSING) or overwrites (OVERWRITE_EXISTING), tracking counts, indices, entries and truncated counts for both. apply_tracking_merge can now apply overwrite merges by replacing target coordinates and updating shared feature columns, while still supporting appending new points. UI changes: merge dialog now exposes a policy combo, preview summary/details and confirmation dialogs were updated and renamed (TrackingMergeReviewDialog) to show either conflicts or overwrites with adjusted wording and buttons. Default behavior remains fill-missing; validation and stale-preview fingerprint checks preserved.

* Allow tracking-result and generic Points as merge targets

Generalize merge target logic and update UI text/selection accordingly.

- LayerLifecycleManager: refine is_mergeable_dlc_points_layer to:
  - disallow config-placeholder layers early
  - validate layer.data shape safely
  - allow tracking-result layers as merge targets (but they do not count as "managed" when require_managed=True)
  - clarify docstring and ordering semantics for iter_mergeable_dlc_points_layers

- Tracking UI (merger.py):
  - update user-facing strings to refer to generic "Points" layers instead of "DLC" layers
  - change target selection logic to exclude the selected source layer from the candidate list
  - adjust the no-target warning text to reflect the broader set of acceptable Points targets

These changes broaden supported merge targets to include tracking-result Points, improve robustness when inspecting layer data, and make the UI wording and selection logic consistent with the new semantics.

* Use median for uniform point size and apply it

Replace nanmean with nanmedian when computing a uniform point size to avoid outlier-driven values. Exported get_uniform_point_size and use it when copying point sizes to a new layer, with a nested fallback to deepcopy(source.size) to preserve robustness if the uniform computation fails.

* Add unit helpers and tests for tracking utils

Introduce pure-unit test helpers in conftest (fake_points_layer_factory, dummy_viewer_factory, tracking_manager_factory, patch_tracking_manager) to avoid requiring a real napari viewer or qtbot. Add a new test module tracking/test_utils.py that exercises napari_deeplabcut.tracking.core.utils: extract_layer_data_and_features, coord/semantic helpers, normalize_points_layer_for_tracking, and tracked naming helpers, covering expected behavior and edge cases.

* Refactor tracking tests; add refine tests

Replace ad-hoc _make_points_layer with a flexible fake_points_layer_factory in tracking tests (conftest) that can produce real napari Points layers or lightweight fake objects and supports explicit features/properties, labels, ids, and extra feature columns. Update test_merge to use the factory, add and adjust assertions for overwrite-aware merge behavior (new overwriteable/overwrites checks and TrackingMergePolicy OVERWRITE_EXISTING tests), and add defensive checks for invalid/stale previews. Add a new test_refine.py containing comprehensive unit tests for preview/apply delete-in-future refine operations. Also add a short header comment to refine.py. These changes improve test coverage and make tests more flexible for both real-layer and pure-unit scenarios.

* Reject 'nan'/'none' labels; clean up tests

Tighten label validation and tidy test files.

- tracking/core/utils.py: Improve normalize_points_layer_for_tracking to treat NaN/None, empty strings, and literal strings 'nan' or 'none' (case-insensitive) as invalid labels by normalizing and explicitly excluding these values.
- _tests/tracking/conftest.py: Replace quoted forward-reference type hints with real types for TrackingWorkerData/TrackingWorkerOutput in DummyTracker methods; minor formatting cleanups.
- _tests/tracking/test_merge.py and test_refine.py: Whitespace and import reordering, trailing-whitespace removal, and ensure final newline; no behavioral test changes.

These changes make label filtering more robust and clean up test code/typing for clarity.

* Move tracking widgets tests

* Add tracking merge UI tests

Add a new test module src/napari_deeplabcut/_tests/tracking/ui/test_merge_ui.py that provides comprehensive unit tests for the tracking merge UI. Tests cover helper utilities (_layer_display_name, preview summary/details), the review dialog, main merge dialog behavior (policies, enablement, summaries), and the workflow (no-candidate warnings, successful merge application, and overwrite confirmation). Uses pytest, qtbot, monkeypatch, and fake_points_layer_factory to simulate layers and preview/apply behaviour.

* Create __init__.py

* Add tests for tracking data utilities

Introduce a new test module covering napari_deeplabcut.tracking.core.data utilities. Tests validate coerce_features_df (returns defensive copy, accepts mappings), add_query_identity_columns (adds tracking columns and preserves originals), expand_query_features_over_time (repeats seed features over frames, sets tracking fields, accepts multiple visibility shapes and errors on mismatch), build_tracking_result_metadata (preserves source metadata, returns deep copy, handles None), and is_tracking_result_points_layer (detects valid tracking metadata and rejects missing/wrong cases).

* Add unit tests for tracking UI behaviors

Add multiple pytest unit tests covering tracking UI behavior and edge cases. New tests verify: no tracking is requested without a video layer; invalid target frames do not emit requests (parametrized for forward/backward); tracking requests are ignored while already tracking; both-way tracking triggers backward tracking only once (single-shot behavior); tracking_finished creates and selects a result keypoints layer; forward/backward absolute/relative controls and sliders stay in sync; and seed query extraction returns only reference-frame keypoints and raises when none are present. Tests use the qtbot fixture and existing test helpers to set up layers and frames.

* Refactor tracking UI tests: fixtures & helpers

Split the monolithic setup_tracking_widget fixture into focused fixtures and helpers to improve test clarity and isolation. Added __future__.annotations import and new helpers: patch_tracking_side_effects (disables plugin hooks), empty_tracking_env, tracking_env, _stub_worker_start, _capture_tracking_requests, plus small refactorings to _set_current_tracking_frame and _put_all_points_on_frame. Tests were converted to use the new fixtures, parameterized where appropriate, reorganized into logical sections, and updated to avoid spawning real worker threads or relying on UI insert hooks. Adjusted assertions and timing/wait logic and consolidated result-layer integration test. Overall this makes the tests less brittle and easier to reason about.

* Add unit tests for tracking models and Cotracker3

Add comprehensive unit tests for tracking models. The new tests (src/.../_tests/tracking/test_models.py) cover register_backbone, TrackingModel initialization and device/model setup, and many Cotracker3 behaviors: prepare_inputs (xy swap without mutating inputs), prepare_outputs (flattening tracks, restoring plugin xy, handling backward tracking/time reversal), and validate_outputs (acceptance and multiple rejection cases for missing metadata, wrong row counts, shape/time/seed-feature mismatches). Also apply a minor comment tweak in models.py suggesting alternate module layout.

* Prevent saving tracking-result Points layers

Add pre-save validation and user feedback for Points layers in save workflow. Introduces helper methods to detect tracking-result layers and validate saveable DLC Points (checks type, tracking-result, config-placeholder, and header). Single-layer saves now block direct saving of tracking-result layers with a warning dialog and log entry; non-DLC/invalid Points show a warning. Multi-layer saves filter candidate layers into tracking and saveable sets, persist folder UI state only for saveable DLC layers, call viewer.layers.save for the selectable set, and return a SaveOutcome that notes any skipped tracking-result layers (with guidance to merge them first).

* Only route DLC saves for DLC Points layers

Make saving behavior more selective: only route a single Points layer through the DLC-specific save path when it is a DeepLabCut-saveable points layer. For tracking-result or foreign/generic Points layers, use the generic napari save instead. Persist folder UI state only for true DLC-saveable layers. Adjust tracking-layer collection and update the post-save status message to reflect that tracking-result layers were saved generically and not as DeepLabCut project data.

* remove redundant checks

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix debug consistency

* Lazy widget imports in factory

* Revert "Merge branch 'cy/tracking-merge-results' of https://github.com/DeepLabCut/napari-deeplabcut into cy/tracking-merge-results"

This reverts commit 0d93660, reversing
changes made to 61955c7.

* Allow saving of tracking-result points layers

Remove the checks that blocked direct saving of tracking-result Points layers and of non-saveable DeepLabCut Points layers so they can be saved via the generic path. Update the final status message to clarify that tracking-result layers are not saved as DeepLabCut project data and will not be reloaded as proper annotations. The promotion/save target flow remains in place.

* Simplify Points post-save handling

Use a generic Points check and managed points list for post-save persistence (isinstance(ly, Points) and layer_manager.managed_points_layers()) instead of DLC-specific saveability checks. Remove the special-case handling and warning for tracking-result layers so saves always return a generic success outcome. This simplifies the save workflow and avoids emitting a tracking-specific message.

* Update `ClickableLabel` in color scheme reference: don't use colored text (#206)

* bugfix: set part_label text color, together with box label

* Parts label text: don't set and clear color; only boldfont

* Refactor ClickableLabel hover and LabelPair color

Introduce a reactive color property and hover-state handling for ClickableLabel: store the pre-hover stylesheet, apply hover color while preserving existing styles (with semicolon handling), and track _hovered state so setting color updates the hover appearance. Call super() in mousePressEvent/enterEvent/leaveEvent to ensure proper event propagation. Add a color property to LabelPair that updates both the color swatch and the nested part label, consolidating previous duplicate logic.

---------

Co-authored-by: C-Achard <cyril.achard@epfl.ch>

* Tracking fixes (#211)

* Sync tracking controls with viewer dims

Replace inline lambda with a dedicated slot and add an initial sync so tracking controls reflect the viewer's current dims on startup. Adds _sync_from_viewer_dims (scheduled once after layout) and _on_current_step_changed to robustly update the reference spinbox and frame controls with exception handling and debug logging. When tracking finishes, capture the viewer's current_step and restore it after creating the tracked layer; _select_keypoint_combo_layer signature now accepts an optional restore_step and will set dims.current_step when selecting the layer.

* Add themed help/info icons and use in widgets

Add a small icon utility and themed SVG assets to standardize help/info buttons across light/dark themes. Introduces src/napari_deeplabcut/ui/icons.py with apply_help_info_icon that picks an appropriate SVG (info_icon_light.svg / info_icon_dark.svg) based on napari appearance settings (with sane fallbacks) and applies icon, size, cursor and tooltip to QToolButton. Replaced ad-hoc QIcon.fromTheme usage in tracking/_widgets.py and layer_stats.py, added two SVG assets, and thread the viewer through KeypointControls -> LayerStatusPanel so the panel can resolve the current theme. This makes help/info buttons consistent across themes and preserves previous sizing/behavior.

* Use indeterminate progress for tracking start

Don't force the UI progress bar to 0 when tracking starts; set the progress bar maximum to 0 to show an indeterminate/busy state and let the model drive progress updates. Update Cotracker3 to invoke progress_callback(0, 0) at start because it doesn't support intermediate progress updates, treating the start as unknown progress until the model reports completion.

* Include SVG info icons in package data

Add info_icon_light.svg and info_icon_dark.svg to MANIFEST.in and update pyproject.toml to include assets/*.svg in the napari_deeplabcut package data. This ensures the SVG icons are packaged and distributed with the project so they are available at runtime.

* Fix trajectory plot x-axis not updating for tracking results (#210)

* Refactor trajectory plotting into plot state

Introduce TrajectorySeries and TrajectoryPlotState dataclasses and refactor the trajectory plotting code to build and render an explicit plot state instead of relying on a widget-global DataFrame. Key changes:

- Add src/napari_deeplabcut/ui/plots/plot_models.py with immutable dataclasses for series and plot state.
- Replace usage of self.df with self._plot_state and store frame bounds and image height in the state.
- Add helper methods: _reset_axes, _frame_values_from_layer_data, _image_height_from_viewer, _frame_bounds_from_x, _build_plot_state, and _render_plot_state to centralize logic for deriving x values, image height, and rendering.
- Improve robustness and logging when building the plot state; surface a viewer status message on failure.
- Simplify and harden color/colormap and plot-mode logic; consolidate plot properties retrieval.
- Use frame-space bounds for axis windowing and use draw_idle for non-blocking redraws.
- Minor header/comment additions.

These changes aim to make plotting more robust, easier to reason about, and fix issues with tracking results plots not being updated accordingly.

* Add df property and adjust tests

Expose the currently plotted DataFrame via a df property that proxies self._plot_state.df (or None when no plot state exists). Add a guard in _df_has_individuals to handle a missing _plot_state. Update tests to stop assigning canvas.df directly and instead reference the plot_state-based df (commented), keeping test semantics while matching the new internal API.

* Fix config segfault from layer removal and overhaul dialog (#208)

* Replace placeholder-config decision flow

Replace the old 'merge' API with a focused placeholder-config decision flow. Introduces PlaceholderConfigAction and PlaceholderConfigDecisionProvider (replacing MergeDecision* types) and updates __init__ exports. Key changes in LayerLifecycleManager: new PointsInsertResult enum, renamed/rewired merge logic to _maybe_merge_config_points_layer -> returns explicit PlaceholderConfigAction or None, _resolve_placeholder_config_action to consult the new provider, refined handling for APPLY_TO_CURRENT / KEEP_AS_SEPARATE_LAYER / CANCEL (including deferred removal and emitting points_layers_merged_requested only on apply), and adjusted layer insert event emission. UI widget updated to set_placeholder_config_decision_provider and to present a three-button dialog asking how to handle added keypoints. Overall this centralizes and clarifies placeholder config handling and fixes problematic layer removal by delaying the deletion slightly.

* Use PlaceholderConfigAction in e2e save test

Import PlaceholderConfigAction and set keypoint_controls.resolve_placeholder_config_action to return APPLY_TO_CURRENT in test_save_e2e.py. This forces the placeholder-config resolution during the config merge step to the intended path.

* Re-export PlaceholderConfigAction and add types

Import PlaceholderConfigAction from the package-level layer_lifecycle module instead of the internal merge submodule, and add explicit type hints to KeypointControls.resolve_placeholder_config_action parameters (Points, tuple[Points, ...], tuple[str, ...], str). Updated the corresponding test import. No functional changes intended—this clarifies the public API and improves static type checking.

* Use constant for delay; improve placeholder logic

Introduce LAYER_REMOVAL_DELAY_MS and replace hard-coded 300ms timers with the constant to avoid magic numbers. Refactor placeholder config resolution by introducing a _default_action helper (chooses APPLY_TO_CURRENT when no added keypoints, otherwise KEEP_AS_SEPARATE_LAYER). Add robust error handling and logging for the placeholder decision provider: catch exceptions, log debug info, validate the provider return type and warn if invalid, and fall back to the default action.

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Use header equality to decide placeholder action

Compare reference and new headers via model_dump() and pass a headers_match flag into the placeholder resolution. Build a clearer message when keypoints were added or when headers differ, and adjust the default action: apply config to the current layer if headers match, otherwise keep as a separate layer. Also rename some local vars for clarity and remove the old one-line message construction.

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix trajectories limits; adhere to requested window when close to boundaries (#213)

---------

Co-authored-by: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com>

* Remove outdated docs

See DeepLabCut/DeepLabCut#3318

* Ensure missing torch is properly handled

* Tag all co authors

Co-Authored-By: Niels <45132115+n-poulsen@users.noreply.github.com>
Co-Authored-By: Bryan Gotti <29003872+brygotti@users.noreply.github.com>
Co-Authored-By: Jun Huang <44811884+junhuang7@users.noreply.github.com>
Co-Authored-By: Riccardo Carpineto <92370991+riccardoprog@users.noreply.github.com>
Co-Authored-By: maud73 <92534343+maud73@users.noreply.github.com>
Co-Authored-By: Alexis Cogne <133367958+alexiscogne@users.noreply.github.com>
Co-Authored-By: Jennifer Ayer <80843866+antigonej@users.noreply.github.com>
Co-Authored-By: Laura Gambaretto <92331544+lauragambaretto@users.noreply.github.com>
Co-Authored-By: Lucas Stoffl <37299767+luczot@users.noreply.github.com>
Co-Authored-By: Aykelia <114958332+aykelia@users.noreply.github.com>

---------

Co-authored-by: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com>
Co-authored-by: Niels <45132115+n-poulsen@users.noreply.github.com>
Co-authored-by: Bryan Gotti <29003872+brygotti@users.noreply.github.com>
Co-authored-by: Jun Huang <44811884+junhuang7@users.noreply.github.com>
Co-authored-by: Riccardo Carpineto <92370991+riccardoprog@users.noreply.github.com>
Co-authored-by: maud73 <92534343+maud73@users.noreply.github.com>
Co-authored-by: Alexis Cogne <133367958+alexiscogne@users.noreply.github.com>
Co-authored-by: Jennifer Ayer <80843866+antigonej@users.noreply.github.com>
Co-authored-by: Laura Gambaretto <92331544+lauragambaretto@users.noreply.github.com>
Co-authored-by: Lucas Stoffl <37299767+luczot@users.noreply.github.com>
Co-authored-by: Aykelia <114958332+aykelia@users.noreply.github.com>
@C-Achard
C-Achard requested review from AlexEMG and MMathisLab May 18, 2026 13:40
C-Achard added a commit to DeepLabCut/napari-deeplabcut that referenced this pull request May 18, 2026
* Create v0_3_1_0.md

* Tracking: rebased and refactored Cotracker integration (#187)

* Add experimental tracking feature (UI, worker, core)

Introduce an experimental point-tracking subsystem: adds a TrackingControls UI, a background TrackingWorker, and core data/models for registering and running trackers. Implements concrete modules under tracking/core (data and abstract TrackingModel with registry), a Qt widget under tracking/_widgets.py, and a worker implementation under tracking/_worker.py. Includes tests (fixtures, widget and worker tests) and a README documenting usage and keybindings; tests register a DummyTracker for unit testing. Adds "torch" to pyproject.toml and updates tox.ini to support the new tracking tests.

Co-Authored-By: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com>

* Restore plugin tracking manifest

Co-Authored-By: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com>

* Make tracking shortcuts configurable & conditional

Introduce configurable tracking shortcuts and only enable them when the tracking widget is visible. Add TRACKING_SHORTCUTS_ENABLED (env NAPARI_DLC_TRACKING_SHORTCUTS_ENABLED, default enabled) to settings, centralize tracking key definitions in config/keybinds.py, and have iter_shortcuts yield tracking shortcuts only when enabled. Update TrackingControls to import those key configs, set an objectName/property for dock detection, and guard bound key callbacks so they only act when the tracking dock is open/visible. Enhance Shortcuts dialog to recognize the new "tracking-points-layer" scope and to hide/disable shortcuts when the tracking widget isn't open. Also update imports (KeypointStore path) and remove duplicate dataclass/keybind definitions.

* Fix shortcut tests

* Improve tracking feature pipeline and result layers

Introduce an identity-preserving tracking data pipeline and create separate tracking result layers. data.py: add tracking metadata constants, convert TrackingWorkerData/Output to use DataFrame for per-query features, and add helpers (coerce_features_df, add_query_identity_columns, expand_query_features_over_time, build_tracking_result_metadata, is_tracking_result_layer) to preserve query identity and expand features over time. models.py: update Cotracker3 to avoid mutating inputs, correctly handle x/y ordering, visibility shapes, time-reversal for backward tracking, and produce flattened keypoints plus expanded per-point feature rows. _widgets.py: add UI helpers to seed query points/features, build a non-destructive tracking result Points layer (with metadata and visual tweaks), wire those into the tracking flow, improve error handling, and select the created layer in the viewer. Overall these changes ensure semantic identity of query points is preserved through inference and that results are stored in a dedicated, annotated layer.

* Fix tracking tests

* Fix incorrect dataclass use

* Improve tracking tests: seed frames and assertions

Add helpers to place all test keypoints on a specific frame and to drive the widget reference frame (_put_all_points_on_frame, _set_current_tracking_frame). Update tests (test_backward_track, test_bothway_track) to use these helpers instead of directly mutating viewer.dims, and add assertions verifying the tracking request payload: correct reference_frame_index, sliced video length, keypoints rebased to local frame 0, original keypoint feature columns (id/name) preserved, and new tracking identity columns (tracking_query_index, tracking_query_frame) populated. Also ensure both-way tracking uses the same seed frame and that forward==reference only triggers backward tracking.

* Remove duplicate worker

* Use queued signals and refine TrackingWorker signals

Switches several signal/slot connections to explicit Qt.QueuedConnection to ensure safe cross-thread delivery and replaces tuple/object-typed signals with primitive signatures. In TrackingControls: trackingRequested now uses Signal(object); worker connections use queued delivery; added _on_worker_started/_on_worker_finished/_on_worker_progress slots, a _request_worker_stop wrapper, and a _debug_thread helper for logging. In TrackingWorker: progress is now Signal(int,int), trackingFinished is Signal(object), track is annotated with Slot(object), and progress.emit now emits two ints; added thread debug logging. These changes improve thread-safety, clarity of signal payloads, and progress reporting.

* Refactor TrackingWorker stop flow and logging

Update tracking worker and UI to improve stop handling, logging, and error reporting.

- TrackingWorker: replace boolean stop flag with threading.Event, add request_stop(), use self.thread for logging, emit trackingStarted/trackingStopped/trackingFinished and finished in appropriate places, avoid QCoreApplication.processEvents, and guard torch.cuda.empty_cache. Ensure early return when stop requested.
- Tests: adapt progress signal handling to accept (current, total) args.
- Widgets: use self.thread attribute in debug logs, derive reference frame index from returned features when available, catch ValueError from seed point queries and show a napari warning instead of crashing, plus small formatting fixes.

These changes make worker shutdown safer across threads, improve diagnostics, and prevent main-thread operations from being performed in the worker.

* Use CoTracker offline model and fix visibility

Switch the CoTracker backend to the offline model and refactor run() to perform whole-clip inference. Inputs.video is converted and permuted to (1, T, C, H, W), queries are batched to (1, K, 3), inference is executed under torch.inference_mode, progress callbacks updated, and a RuntimeError is raised if no predictions are returned. Also add early-stop handling (return None). Additionally, fix visibility array handling in expand_query_features_over_time by squeezing a leading singleton channel dimension when vis.ndim == 3 and vis.shape[0] == 1.

* Adjust tracking UI visuals and threading bugfix

Tweak tracking UI appearance and fix a threading/logging bug.

- tracking/_widgets.py: add TODO about slider sync; change tracking layer visuals to use symbol="cross", opacity=0.85, and border_color="green" to better distinguish results.
- tracking/ui/worker.py: fix logging call by using self.thread instead of self.thread() to avoid a mistaken call in the thread information.

* Update src/napari_deeplabcut/tracking/ui/worker.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/napari_deeplabcut/tracking/core/models.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update src/napari_deeplabcut/_tests/config/test_keybinds.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Improve tracking widgets and lazy torch import

Use a lifecycle manager in TrackingControls and tighten viewer typing; update test helper to locate the TrackingControls dock by isinstance and add TYPE_CHECKING napari types. Fix a TYPE_CHECKING import in tracking models (correct TrackingModel import). Replace top-level torch import in the tracking worker with a lazy import and a clear ImportError message advising how to install the tracking extras, and improve CUDA cache cleanup logging. These changes reduce heavy top-level imports, improve typing correctness, and make missing-dependency errors clearer.

* Run pre-commit

* Tracking UI: Merge tracking results (#190)

* Add experimental tracking feature (UI, worker, core)

Introduce an experimental point-tracking subsystem: adds a TrackingControls UI, a background TrackingWorker, and core data/models for registering and running trackers. Implements concrete modules under tracking/core (data and abstract TrackingModel with registry), a Qt widget under tracking/_widgets.py, and a worker implementation under tracking/_worker.py. Includes tests (fixtures, widget and worker tests) and a README documenting usage and keybindings; tests register a DummyTracker for unit testing. Adds "torch" to pyproject.toml and updates tox.ini to support the new tracking tests.

Co-Authored-By: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com>

* Add tracking merge logic for Points layers

Introduce src/napari_deeplabcut/tracking/core/merge.py implementing merge preview and apply workflows for merging tracking-result Points into DLC Points layers. Adds data models (TrackingMergePreview, LayerFingerprint, TrackingMergeConflictEntry) and core APIs preview_tracking_merge and apply_tracking_merge with staleness checks via fingerprints, conflict detection, and a FILL_MISSING policy. Includes robust normalization/validation helpers for Napari Points data/features, coordinate-tolerance comparisons, duplicate-slot detection, and safe feature merging that preserves target schema.

* Support tracking-result Points layers

Add lifecycle manager support for DLC tracking-result Points layers: import the tracking metadata helpers and expose methods to read tracking metadata, classify tracking-result layers, extract source layer names, detect mergeable DLC Points layers, iterate tracking/mergeable layers, and suggest a default merge target. These methods centralize viewer/session-facing semantics for tracking results and provide merge-selection logic (managed vs live, name matching, active layer preference). Also rename the helper in tracking core from is_tracking_result_layer to is_tracking_result_points_layer and update imports accordingly.

* Add tracking merge UI and workflow

Introduce a new tracking merge UI and workflow to merge tracked points into DLC Points layers. Adds a new module tracking/ui/merger.py implementing TrackingMergeWorkflow, TrackingMergeDialog, and TrackingMergeConflictsDialog with preview, conflict reporting, and application logic (uses preview_tracking_merge and apply_tracking_merge). Integrates a "Merge tracked points…" button into TrackingControls (_widgets.py), wiring it to open the workflow (with a hinted active source), and adds the button to the controls layout. The workflow validates candidates, shows transactional dialogs, applies the merge to the target layer, updates the viewer selection/status, and marks layer presentation changes with error handling and user-facing warnings.

* Add tests for tracking merge behaviors

Add comprehensive tests for tracking merge utilities: fingerprint_points_layer, preview_tracking_merge, and apply_tracking_merge. Tests cover valid/invalid previews (same layer, duplicate semantic slots), classification of identical/conflict/appendable/invalid rows, preservation of target schema and prevention of source-only column leakage, handling of no-op merges (returns copies), detection of stale previews when layer name or feature columns change, use of id in semantic identity, and coordinate-tolerance effects on identical vs conflict. Also add a small header comment to the tracking conftest.

* Restore changes from parent branch

* Include preview reason in invalid merge error

Improve diagnostics when applying an invalid tracking merge preview by appending preview.invalid_reason to the raised ValueError (defaults to 'Unknown reason.'). This change makes the failure message more informative without altering control flow or behavior.

* Escape layer names in merge UI, simplify logs

HTML-escape source/target layer display names shown in the tracking merge dialogs to avoid HTML injection and rendering issues (adds html.escape import). Also remove explicit exc_info argument passed to logger.exception calls, relying on logger.exception to include exception info by default.

* Validate coordinate columns before merge

Add a pre-merge validation that ensures the source (append_df) contains all coordinate columns required by the target layer. If any coord_* columns are missing, the code now raises a ValueError with a clear message listing the missing required columns and the coord_* columns present in the source, and suggests refreshing the preview or merging into a compatible target. This prevents silent dimensionality mismatches and potential data corruption during tracking merges.

* Add review-only mode to merge conflicts dialog

Introduce a review_only mode to TrackingMergeConflictsDialog so it can be used both as an informational "Review conflicts" viewer and as a confirmation dialog before applying a partial merge. Adds a review_only parameter, adjusts window titles and summary text, and changes the button row (Close for review mode; Cancel/Merge for confirm mode). Exposes a new static review(...) helper and calls it from the preview path. Also simplifies label text construction by using preview.source_layer_name/target_layer_name directly.

* Make merge button more distinct

* Handle tracking-result points in trajectory plot

Integrate the LayerLifecycleManager with the trajectory canvas and improve selection of plottable Points layers. Add manager methods (is_plottable_traj_layer, iter_plottable_traj_layers, suggest_plottable_traj_layer) to determine and suggest eligible layers in viewer order, prioritizing active and managed layers. Have TrajectoryMatplotlibCanvas use get_or_create_layer_manager, refresh on layer insert/remove/selection events, and ask the manager for the plot layer. Add logic to merge tracking-result layer.features and properties into a properties mapping so tracking outputs can be plotted, and use that mapping when building the DataFrame for plotting. Add defensive checks and debug logs to avoid plotting incomplete or incompatible Points layers.

* Standardize logger names to module __name__

Replace hard-coded logger strings with logging.getLogger(__name__) in several modules (widgets, layer lifecycle manager and registry) and update tests and conftest to use the underscore package name 'napari_deeplabcut' instead of the hyphen variant. This aligns logger names with module __name__ and keeps logging configuration and test expectations consistent; no functional behavior changes intended.

* Add future-delete refine, utils; refactor merge

Introduce tracking refine and utility modules and wire a new UI action to delete tracked points in future frames.

- Add tracking/core/refine.py: preview_delete_tracking_points_in_future and apply_delete_tracking_points_in_future plus TrackingFutureDeletePreview dataclass to support previewing and applying bulk deletion of matching tracked points in future frames.
- Add tracking/core/utils.py: utilities for robust Points layer handling (extract_layer_data_and_features, coord_columns_for_data, pick_semantic_series, coerce_frame_series, normalize_slot_id, slot key helpers, normalize_points_layer_for_tracking) and helpers for generating stable tracked-layer names (make_tracking_iteration_name and related helpers).
- Update widgets (_widgets.py): add a "Delete selected points in future frames" button, preview/apply flow with confirmations and notifications, improved error messages, use make_tracking_iteration_name for new tracked layers, and minor UI/log cleanup.
- Refactor tracking/core/merge.py to reuse the new utils (coord/normalize/extract/format helpers), remove duplicated helper implementations, and adapt preview/apply logic to new utility APIs.

These changes centralize Points data/feature normalization and naming logic, add a user-facing refine action for tracked predictions, and reduce duplication across merge/refine code paths.

* Refactor tracking name generation & add typing

Add TYPE_CHECKING imports and napari type hints; update spawn.py viewer helpers to use forward-referenced napari.Viewer types (non-functional). Refactor tracking utils to accept an explicit LayerLifecycleManager instead of relying on self, move get_or_create_layer_manager import into the module, and derive a family regex via _tracking_version_family_pattern so tracked-layer versions are incremented per (source, tracker) family regardless of reference frame. Small cleanup and formatting adjustments.

* Add deferred keypoint combo selection with QTimer

Import QTimer and add _select_keypoint_combo_layer to TrackingControls. The new helper resets the keypoint combo choices and schedules selecting a newly added Points layer on the next event-loop tick to avoid race conditions during layer insertion. Includes robust try/except logging around choice reset and selection, and leaves a FIXME about the unowned QTimer usage.

* Update _widgets.py

* Update widget_factory.py

* Run pre-commit

* Add overwrite-existing merge policy

Introduce a new TrackingMergePolicy.OVERWRITE_EXISTING and extend the merge preview/apply workflow to support overwriting existing target points. preview_tracking_merge now classifies mismatches as either conflicts (FILL_MISSING) or overwrites (OVERWRITE_EXISTING), tracking counts, indices, entries and truncated counts for both. apply_tracking_merge can now apply overwrite merges by replacing target coordinates and updating shared feature columns, while still supporting appending new points. UI changes: merge dialog now exposes a policy combo, preview summary/details and confirmation dialogs were updated and renamed (TrackingMergeReviewDialog) to show either conflicts or overwrites with adjusted wording and buttons. Default behavior remains fill-missing; validation and stale-preview fingerprint checks preserved.

* Allow tracking-result and generic Points as merge targets

Generalize merge target logic and update UI text/selection accordingly.

- LayerLifecycleManager: refine is_mergeable_dlc_points_layer to:
  - disallow config-placeholder layers early
  - validate layer.data shape safely
  - allow tracking-result layers as merge targets (but they do not count as "managed" when require_managed=True)
  - clarify docstring and ordering semantics for iter_mergeable_dlc_points_layers

- Tracking UI (merger.py):
  - update user-facing strings to refer to generic "Points" layers instead of "DLC" layers
  - change target selection logic to exclude the selected source layer from the candidate list
  - adjust the no-target warning text to reflect the broader set of acceptable Points targets

These changes broaden supported merge targets to include tracking-result Points, improve robustness when inspecting layer data, and make the UI wording and selection logic consistent with the new semantics.

* Use median for uniform point size and apply it

Replace nanmean with nanmedian when computing a uniform point size to avoid outlier-driven values. Exported get_uniform_point_size and use it when copying point sizes to a new layer, with a nested fallback to deepcopy(source.size) to preserve robustness if the uniform computation fails.

* Add unit helpers and tests for tracking utils

Introduce pure-unit test helpers in conftest (fake_points_layer_factory, dummy_viewer_factory, tracking_manager_factory, patch_tracking_manager) to avoid requiring a real napari viewer or qtbot. Add a new test module tracking/test_utils.py that exercises napari_deeplabcut.tracking.core.utils: extract_layer_data_and_features, coord/semantic helpers, normalize_points_layer_for_tracking, and tracked naming helpers, covering expected behavior and edge cases.

* Refactor tracking tests; add refine tests

Replace ad-hoc _make_points_layer with a flexible fake_points_layer_factory in tracking tests (conftest) that can produce real napari Points layers or lightweight fake objects and supports explicit features/properties, labels, ids, and extra feature columns. Update test_merge to use the factory, add and adjust assertions for overwrite-aware merge behavior (new overwriteable/overwrites checks and TrackingMergePolicy OVERWRITE_EXISTING tests), and add defensive checks for invalid/stale previews. Add a new test_refine.py containing comprehensive unit tests for preview/apply delete-in-future refine operations. Also add a short header comment to refine.py. These changes improve test coverage and make tests more flexible for both real-layer and pure-unit scenarios.

* Reject 'nan'/'none' labels; clean up tests

Tighten label validation and tidy test files.

- tracking/core/utils.py: Improve normalize_points_layer_for_tracking to treat NaN/None, empty strings, and literal strings 'nan' or 'none' (case-insensitive) as invalid labels by normalizing and explicitly excluding these values.
- _tests/tracking/conftest.py: Replace quoted forward-reference type hints with real types for TrackingWorkerData/TrackingWorkerOutput in DummyTracker methods; minor formatting cleanups.
- _tests/tracking/test_merge.py and test_refine.py: Whitespace and import reordering, trailing-whitespace removal, and ensure final newline; no behavioral test changes.

These changes make label filtering more robust and clean up test code/typing for clarity.

* Move tracking widgets tests

* Add tracking merge UI tests

Add a new test module src/napari_deeplabcut/_tests/tracking/ui/test_merge_ui.py that provides comprehensive unit tests for the tracking merge UI. Tests cover helper utilities (_layer_display_name, preview summary/details), the review dialog, main merge dialog behavior (policies, enablement, summaries), and the workflow (no-candidate warnings, successful merge application, and overwrite confirmation). Uses pytest, qtbot, monkeypatch, and fake_points_layer_factory to simulate layers and preview/apply behaviour.

* Create __init__.py

* Add tests for tracking data utilities

Introduce a new test module covering napari_deeplabcut.tracking.core.data utilities. Tests validate coerce_features_df (returns defensive copy, accepts mappings), add_query_identity_columns (adds tracking columns and preserves originals), expand_query_features_over_time (repeats seed features over frames, sets tracking fields, accepts multiple visibility shapes and errors on mismatch), build_tracking_result_metadata (preserves source metadata, returns deep copy, handles None), and is_tracking_result_points_layer (detects valid tracking metadata and rejects missing/wrong cases).

* Add unit tests for tracking UI behaviors

Add multiple pytest unit tests covering tracking UI behavior and edge cases. New tests verify: no tracking is requested without a video layer; invalid target frames do not emit requests (parametrized for forward/backward); tracking requests are ignored while already tracking; both-way tracking triggers backward tracking only once (single-shot behavior); tracking_finished creates and selects a result keypoints layer; forward/backward absolute/relative controls and sliders stay in sync; and seed query extraction returns only reference-frame keypoints and raises when none are present. Tests use the qtbot fixture and existing test helpers to set up layers and frames.

* Refactor tracking UI tests: fixtures & helpers

Split the monolithic setup_tracking_widget fixture into focused fixtures and helpers to improve test clarity and isolation. Added __future__.annotations import and new helpers: patch_tracking_side_effects (disables plugin hooks), empty_tracking_env, tracking_env, _stub_worker_start, _capture_tracking_requests, plus small refactorings to _set_current_tracking_frame and _put_all_points_on_frame. Tests were converted to use the new fixtures, parameterized where appropriate, reorganized into logical sections, and updated to avoid spawning real worker threads or relying on UI insert hooks. Adjusted assertions and timing/wait logic and consolidated result-layer integration test. Overall this makes the tests less brittle and easier to reason about.

* Add unit tests for tracking models and Cotracker3

Add comprehensive unit tests for tracking models. The new tests (src/.../_tests/tracking/test_models.py) cover register_backbone, TrackingModel initialization and device/model setup, and many Cotracker3 behaviors: prepare_inputs (xy swap without mutating inputs), prepare_outputs (flattening tracks, restoring plugin xy, handling backward tracking/time reversal), and validate_outputs (acceptance and multiple rejection cases for missing metadata, wrong row counts, shape/time/seed-feature mismatches). Also apply a minor comment tweak in models.py suggesting alternate module layout.

* Prevent saving tracking-result Points layers

Add pre-save validation and user feedback for Points layers in save workflow. Introduces helper methods to detect tracking-result layers and validate saveable DLC Points (checks type, tracking-result, config-placeholder, and header). Single-layer saves now block direct saving of tracking-result layers with a warning dialog and log entry; non-DLC/invalid Points show a warning. Multi-layer saves filter candidate layers into tracking and saveable sets, persist folder UI state only for saveable DLC layers, call viewer.layers.save for the selectable set, and return a SaveOutcome that notes any skipped tracking-result layers (with guidance to merge them first).

* Only route DLC saves for DLC Points layers

Make saving behavior more selective: only route a single Points layer through the DLC-specific save path when it is a DeepLabCut-saveable points layer. For tracking-result or foreign/generic Points layers, use the generic napari save instead. Persist folder UI state only for true DLC-saveable layers. Adjust tracking-layer collection and update the post-save status message to reflect that tracking-result layers were saved generically and not as DeepLabCut project data.

* remove redundant checks

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix debug consistency

* Lazy widget imports in factory

* Revert "Merge branch 'cy/tracking-merge-results' of https://github.com/DeepLabCut/napari-deeplabcut into cy/tracking-merge-results"

This reverts commit 0d93660, reversing
changes made to 61955c7.

* Allow saving of tracking-result points layers

Remove the checks that blocked direct saving of tracking-result Points layers and of non-saveable DeepLabCut Points layers so they can be saved via the generic path. Update the final status message to clarify that tracking-result layers are not saved as DeepLabCut project data and will not be reloaded as proper annotations. The promotion/save target flow remains in place.

* Simplify Points post-save handling

Use a generic Points check and managed points list for post-save persistence (isinstance(ly, Points) and layer_manager.managed_points_layers()) instead of DLC-specific saveability checks. Remove the special-case handling and warning for tracking-result layers so saves always return a generic success outcome. This simplifies the save workflow and avoids emitting a tracking-specific message.

* Update `ClickableLabel` in color scheme reference: don't use colored text (#206)

* bugfix: set part_label text color, together with box label

* Parts label text: don't set and clear color; only boldfont

* Refactor ClickableLabel hover and LabelPair color

Introduce a reactive color property and hover-state handling for ClickableLabel: store the pre-hover stylesheet, apply hover color while preserving existing styles (with semicolon handling), and track _hovered state so setting color updates the hover appearance. Call super() in mousePressEvent/enterEvent/leaveEvent to ensure proper event propagation. Add a color property to LabelPair that updates both the color swatch and the nested part label, consolidating previous duplicate logic.

---------

Co-authored-by: C-Achard <cyril.achard@epfl.ch>

* Tracking fixes (#211)

* Sync tracking controls with viewer dims

Replace inline lambda with a dedicated slot and add an initial sync so tracking controls reflect the viewer's current dims on startup. Adds _sync_from_viewer_dims (scheduled once after layout) and _on_current_step_changed to robustly update the reference spinbox and frame controls with exception handling and debug logging. When tracking finishes, capture the viewer's current_step and restore it after creating the tracked layer; _select_keypoint_combo_layer signature now accepts an optional restore_step and will set dims.current_step when selecting the layer.

* Add themed help/info icons and use in widgets

Add a small icon utility and themed SVG assets to standardize help/info buttons across light/dark themes. Introduces src/napari_deeplabcut/ui/icons.py with apply_help_info_icon that picks an appropriate SVG (info_icon_light.svg / info_icon_dark.svg) based on napari appearance settings (with sane fallbacks) and applies icon, size, cursor and tooltip to QToolButton. Replaced ad-hoc QIcon.fromTheme usage in tracking/_widgets.py and layer_stats.py, added two SVG assets, and thread the viewer through KeypointControls -> LayerStatusPanel so the panel can resolve the current theme. This makes help/info buttons consistent across themes and preserves previous sizing/behavior.

* Use indeterminate progress for tracking start

Don't force the UI progress bar to 0 when tracking starts; set the progress bar maximum to 0 to show an indeterminate/busy state and let the model drive progress updates. Update Cotracker3 to invoke progress_callback(0, 0) at start because it doesn't support intermediate progress updates, treating the start as unknown progress until the model reports completion.

* Include SVG info icons in package data

Add info_icon_light.svg and info_icon_dark.svg to MANIFEST.in and update pyproject.toml to include assets/*.svg in the napari_deeplabcut package data. This ensures the SVG icons are packaged and distributed with the project so they are available at runtime.

* Fix trajectory plot x-axis not updating for tracking results (#210)

* Refactor trajectory plotting into plot state

Introduce TrajectorySeries and TrajectoryPlotState dataclasses and refactor the trajectory plotting code to build and render an explicit plot state instead of relying on a widget-global DataFrame. Key changes:

- Add src/napari_deeplabcut/ui/plots/plot_models.py with immutable dataclasses for series and plot state.
- Replace usage of self.df with self._plot_state and store frame bounds and image height in the state.
- Add helper methods: _reset_axes, _frame_values_from_layer_data, _image_height_from_viewer, _frame_bounds_from_x, _build_plot_state, and _render_plot_state to centralize logic for deriving x values, image height, and rendering.
- Improve robustness and logging when building the plot state; surface a viewer status message on failure.
- Simplify and harden color/colormap and plot-mode logic; consolidate plot properties retrieval.
- Use frame-space bounds for axis windowing and use draw_idle for non-blocking redraws.
- Minor header/comment additions.

These changes aim to make plotting more robust, easier to reason about, and fix issues with tracking results plots not being updated accordingly.

* Add df property and adjust tests

Expose the currently plotted DataFrame via a df property that proxies self._plot_state.df (or None when no plot state exists). Add a guard in _df_has_individuals to handle a missing _plot_state. Update tests to stop assigning canvas.df directly and instead reference the plot_state-based df (commented), keeping test semantics while matching the new internal API.

* Fix config segfault from layer removal and overhaul dialog (#208)

* Replace placeholder-config decision flow

Replace the old 'merge' API with a focused placeholder-config decision flow. Introduces PlaceholderConfigAction and PlaceholderConfigDecisionProvider (replacing MergeDecision* types) and updates __init__ exports. Key changes in LayerLifecycleManager: new PointsInsertResult enum, renamed/rewired merge logic to _maybe_merge_config_points_layer -> returns explicit PlaceholderConfigAction or None, _resolve_placeholder_config_action to consult the new provider, refined handling for APPLY_TO_CURRENT / KEEP_AS_SEPARATE_LAYER / CANCEL (including deferred removal and emitting points_layers_merged_requested only on apply), and adjusted layer insert event emission. UI widget updated to set_placeholder_config_decision_provider and to present a three-button dialog asking how to handle added keypoints. Overall this centralizes and clarifies placeholder config handling and fixes problematic layer removal by delaying the deletion slightly.

* Use PlaceholderConfigAction in e2e save test

Import PlaceholderConfigAction and set keypoint_controls.resolve_placeholder_config_action to return APPLY_TO_CURRENT in test_save_e2e.py. This forces the placeholder-config resolution during the config merge step to the intended path.

* Re-export PlaceholderConfigAction and add types

Import PlaceholderConfigAction from the package-level layer_lifecycle module instead of the internal merge submodule, and add explicit type hints to KeypointControls.resolve_placeholder_config_action parameters (Points, tuple[Points, ...], tuple[str, ...], str). Updated the corresponding test import. No functional changes intended—this clarifies the public API and improves static type checking.

* Use constant for delay; improve placeholder logic

Introduce LAYER_REMOVAL_DELAY_MS and replace hard-coded 300ms timers with the constant to avoid magic numbers. Refactor placeholder config resolution by introducing a _default_action helper (chooses APPLY_TO_CURRENT when no added keypoints, otherwise KEEP_AS_SEPARATE_LAYER). Add robust error handling and logging for the placeholder decision provider: catch exceptions, log debug info, validate the provider return type and warn if invalid, and fall back to the default action.

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Use header equality to decide placeholder action

Compare reference and new headers via model_dump() and pass a headers_match flag into the placeholder resolution. Build a clearer message when keypoints were added or when headers differ, and adjust the default action: apply config to the current layer if headers match, otherwise keep as a separate layer. Also rename some local vars for clarity and remove the old one-line message construction.

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix trajectories limits; adhere to requested window when close to boundaries (#213)

---------

Co-authored-by: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com>

* Remove outdated docs

See DeepLabCut/DeepLabCut#3318

* Ensure missing torch is properly handled

* Tag all co authors

Co-Authored-By: Niels <45132115+n-poulsen@users.noreply.github.com>
Co-Authored-By: Bryan Gotti <29003872+brygotti@users.noreply.github.com>
Co-Authored-By: Jun Huang <44811884+junhuang7@users.noreply.github.com>
Co-Authored-By: Riccardo Carpineto <92370991+riccardoprog@users.noreply.github.com>
Co-Authored-By: maud73 <92534343+maud73@users.noreply.github.com>
Co-Authored-By: Alexis Cogne <133367958+alexiscogne@users.noreply.github.com>
Co-Authored-By: Jennifer Ayer <80843866+antigonej@users.noreply.github.com>
Co-Authored-By: Laura Gambaretto <92331544+lauragambaretto@users.noreply.github.com>
Co-Authored-By: Lucas Stoffl <37299767+luczot@users.noreply.github.com>
Co-Authored-By: Aykelia <114958332+aykelia@users.noreply.github.com>

---------

Co-authored-by: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com>
Co-authored-by: Niels <45132115+n-poulsen@users.noreply.github.com>
Co-authored-by: Bryan Gotti <29003872+brygotti@users.noreply.github.com>
Co-authored-by: Jun Huang <44811884+junhuang7@users.noreply.github.com>
Co-authored-by: Riccardo Carpineto <92370991+riccardoprog@users.noreply.github.com>
Co-authored-by: maud73 <92534343+maud73@users.noreply.github.com>
Co-authored-by: Alexis Cogne <133367958+alexiscogne@users.noreply.github.com>
Co-authored-by: Jennifer Ayer <80843866+antigonej@users.noreply.github.com>
Co-authored-by: Laura Gambaretto <92331544+lauragambaretto@users.noreply.github.com>
Co-authored-by: Lucas Stoffl <37299767+luczot@users.noreply.github.com>
Co-authored-by: Aykelia <114958332+aykelia@users.noreply.github.com>

---------

Co-authored-by: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com>
Co-authored-by: Niels <45132115+n-poulsen@users.noreply.github.com>
Co-authored-by: Bryan Gotti <29003872+brygotti@users.noreply.github.com>
Co-authored-by: Jun Huang <44811884+junhuang7@users.noreply.github.com>
Co-authored-by: Riccardo Carpineto <92370991+riccardoprog@users.noreply.github.com>
Co-authored-by: maud73 <92534343+maud73@users.noreply.github.com>
Co-authored-by: Alexis Cogne <133367958+alexiscogne@users.noreply.github.com>
Co-authored-by: Jennifer Ayer <80843866+antigonej@users.noreply.github.com>
Co-authored-by: Laura Gambaretto <92331544+lauragambaretto@users.noreply.github.com>
Co-authored-by: Lucas Stoffl <37299767+luczot@users.noreply.github.com>
Co-authored-by: Aykelia <114958332+aykelia@users.noreply.github.com>
@C-Achard

Copy link
Copy Markdown
Collaborator Author

napari CoTracker is live, would be great to release this whenever we can!

Update napari documentation to improve clarity and guidance for tracking and labeling workflows. Changes include:

- Add a section anchor for the DLC basic workflow in docs/gui/napari/basic_usage.md.
- Expand docs/gui/napari/tracking/basic_usage.md to cover loading extracted frames, reference the DLC basic workflow, and add a note about PyTorch/GPU requirements for the [tracking] extra.
- Clarify how tracking is initialized (only from the selected reference frame), background behavior, shortcut visibility, and visual styling/accessibility notes.
- Add overwrite warnings, better explanation of deleting future tracked points, and explicit guidance on merging and saving (tracking layers are intermediate; tracking layers saved as CSVs are not written back to CollectedData_*.h5).
- Recommend running tracking on original videos for continuity and adjust several wording/grammar and link improvements throughout the tracking docs.

Files changed: docs/gui/napari/basic_usage.md, docs/gui/napari/tracking/basic_usage.md.
@MMathisLab
MMathisLab merged commit 835a3a3 into cy/docs-audit-2026-toc-update May 21, 2026
3 checks passed
@MMathisLab
MMathisLab deleted the cy/docs-update-napari-tracking branch May 21, 2026 09:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation documentation updates/comments napari-dlc Related to napari-deeplabcut (labeling workflow, crop, frame extraction...) new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants