Documentation: napari-deeplabcut CoTracker integration - #3318
Merged
MMathisLab merged 13 commits intoMay 21, 2026
Conversation
Shorten top-level headings in the napari documentation by removing the redundant "napari-DLC -" prefix for consistency and brevity. Updated docs/gui/napari/basic_usage.md and docs/gui/napari/advanced_usage.md to use simpler headings.
Add a new user guide page for the Tracking Controls widget (docs/gui/napari/tracking/basic_usage.md) describing UI, workflow, requirements, tracking actions, refinement/merge tools, model attribution (CoTracker3), troubleshooting, and limitations. Also add the accompanying controls image (docs/images/napari/tracking/controls.png). These additions document how to run point tracking, refine results, and merge tracked points back into DeepLabCut projects.
6 tasks
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new documentation page describing the napari-deeplabcut automated point tracking workflow (CoTracker integration) and wires it into the docs navigation, plus a small cleanup to existing napari page headings.
Changes:
- Added a new napari tracking user guide page covering requirements, UI walkthrough, workflow, troubleshooting, and model attribution/limitations.
- Updated
_toc.ymlto include the new tracking page under the napari section. - Simplified the titles of existing napari “basic” and “advanced” usage pages.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| docs/gui/napari/tracking/basic_usage.md | New end-to-end tracking workflow documentation and model attribution section. |
| docs/gui/napari/basic_usage.md | Heading/title cleanup for the basic usage page. |
| docs/gui/napari/advanced_usage.md | Heading/title cleanup for the advanced usage page. |
| _toc.yml | Adds the tracking docs page to the napari documentation TOC. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Update docs/gui/napari/tracking/basic_usage.md to improve clarity and fix typos: rename "Keypoint Controls layer" to "Keypoint Controls dock widget", correct "Emprical" to "Empirical", and small whitespace/wording tweaks. Restructure the Limitations/Future directions area by adding "Important considerations" and "Future features" headings, move and reword the CoTracker3 note into Future features, consolidate and bullet manual curation and training-set imbalance guidance, and remove duplicate lines. These changes improve readability and provide clearer guidance for users.
Collaborator
Author
|
Do we want more images ? |
Clean up docs/gui/napari/tracking/basic_usage.md: add missing blank line in a tip, normalize bullet indentation and spacing, fix inconsistent numbered lists (renumber steps and convert some items to sub-bullets), and improve clarity in workflow instructions. Also add a brief consideration that manual annotation can sometimes be faster than heavy tracking corrections and note preference for continuous frames. These are purely documentation/formatting edits to improve readability and usability.
Revise and clarify the Tracking Controls documentation: wrap third-party model attribution in a note, add a brief reminder that tracking accelerates but does not replace manual review, and split Requirements into “In napari” and “In your Python environment”. Improve UI documentation with a control/description table, a figure placeholder, and a note about available models. Reword and tighten many instructions (frame/reference wording, tracking range, actions, keyboard shortcuts), clarify that tracking runs in a background worker, and explain that tracking-result layers are intermediate and must be merged/saved to update DLC project files. Add a prominent warning that there is currently no undo for deletion/merge operations. Minor formatting and grammar fixes throughout (CoTracker attribution links, troubleshooting bullets, phrasing) to improve readability and consistency.
C-Achard
added a commit
to DeepLabCut/napari-deeplabcut
that referenced
this pull request
May 18, 2026
C-Achard
marked this pull request as ready for review
May 18, 2026 11:06
C-Achard
added a commit
to DeepLabCut/napari-deeplabcut
that referenced
this pull request
May 18, 2026
* Add experimental tracking feature (UI, worker, core) Introduce an experimental point-tracking subsystem: adds a TrackingControls UI, a background TrackingWorker, and core data/models for registering and running trackers. Implements concrete modules under tracking/core (data and abstract TrackingModel with registry), a Qt widget under tracking/_widgets.py, and a worker implementation under tracking/_worker.py. Includes tests (fixtures, widget and worker tests) and a README documenting usage and keybindings; tests register a DummyTracker for unit testing. Adds "torch" to pyproject.toml and updates tox.ini to support the new tracking tests. Co-Authored-By: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com> * Restore plugin tracking manifest Co-Authored-By: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com> * Make tracking shortcuts configurable & conditional Introduce configurable tracking shortcuts and only enable them when the tracking widget is visible. Add TRACKING_SHORTCUTS_ENABLED (env NAPARI_DLC_TRACKING_SHORTCUTS_ENABLED, default enabled) to settings, centralize tracking key definitions in config/keybinds.py, and have iter_shortcuts yield tracking shortcuts only when enabled. Update TrackingControls to import those key configs, set an objectName/property for dock detection, and guard bound key callbacks so they only act when the tracking dock is open/visible. Enhance Shortcuts dialog to recognize the new "tracking-points-layer" scope and to hide/disable shortcuts when the tracking widget isn't open. Also update imports (KeypointStore path) and remove duplicate dataclass/keybind definitions. * Fix shortcut tests * Improve tracking feature pipeline and result layers Introduce an identity-preserving tracking data pipeline and create separate tracking result layers. data.py: add tracking metadata constants, convert TrackingWorkerData/Output to use DataFrame for per-query features, and add helpers (coerce_features_df, add_query_identity_columns, expand_query_features_over_time, build_tracking_result_metadata, is_tracking_result_layer) to preserve query identity and expand features over time. models.py: update Cotracker3 to avoid mutating inputs, correctly handle x/y ordering, visibility shapes, time-reversal for backward tracking, and produce flattened keypoints plus expanded per-point feature rows. _widgets.py: add UI helpers to seed query points/features, build a non-destructive tracking result Points layer (with metadata and visual tweaks), wire those into the tracking flow, improve error handling, and select the created layer in the viewer. Overall these changes ensure semantic identity of query points is preserved through inference and that results are stored in a dedicated, annotated layer. * Fix tracking tests * Fix incorrect dataclass use * Improve tracking tests: seed frames and assertions Add helpers to place all test keypoints on a specific frame and to drive the widget reference frame (_put_all_points_on_frame, _set_current_tracking_frame). Update tests (test_backward_track, test_bothway_track) to use these helpers instead of directly mutating viewer.dims, and add assertions verifying the tracking request payload: correct reference_frame_index, sliced video length, keypoints rebased to local frame 0, original keypoint feature columns (id/name) preserved, and new tracking identity columns (tracking_query_index, tracking_query_frame) populated. Also ensure both-way tracking uses the same seed frame and that forward==reference only triggers backward tracking. * Remove duplicate worker * Use queued signals and refine TrackingWorker signals Switches several signal/slot connections to explicit Qt.QueuedConnection to ensure safe cross-thread delivery and replaces tuple/object-typed signals with primitive signatures. In TrackingControls: trackingRequested now uses Signal(object); worker connections use queued delivery; added _on_worker_started/_on_worker_finished/_on_worker_progress slots, a _request_worker_stop wrapper, and a _debug_thread helper for logging. In TrackingWorker: progress is now Signal(int,int), trackingFinished is Signal(object), track is annotated with Slot(object), and progress.emit now emits two ints; added thread debug logging. These changes improve thread-safety, clarity of signal payloads, and progress reporting. * Refactor TrackingWorker stop flow and logging Update tracking worker and UI to improve stop handling, logging, and error reporting. - TrackingWorker: replace boolean stop flag with threading.Event, add request_stop(), use self.thread for logging, emit trackingStarted/trackingStopped/trackingFinished and finished in appropriate places, avoid QCoreApplication.processEvents, and guard torch.cuda.empty_cache. Ensure early return when stop requested. - Tests: adapt progress signal handling to accept (current, total) args. - Widgets: use self.thread attribute in debug logs, derive reference frame index from returned features when available, catch ValueError from seed point queries and show a napari warning instead of crashing, plus small formatting fixes. These changes make worker shutdown safer across threads, improve diagnostics, and prevent main-thread operations from being performed in the worker. * Use CoTracker offline model and fix visibility Switch the CoTracker backend to the offline model and refactor run() to perform whole-clip inference. Inputs.video is converted and permuted to (1, T, C, H, W), queries are batched to (1, K, 3), inference is executed under torch.inference_mode, progress callbacks updated, and a RuntimeError is raised if no predictions are returned. Also add early-stop handling (return None). Additionally, fix visibility array handling in expand_query_features_over_time by squeezing a leading singleton channel dimension when vis.ndim == 3 and vis.shape[0] == 1. * Adjust tracking UI visuals and threading bugfix Tweak tracking UI appearance and fix a threading/logging bug. - tracking/_widgets.py: add TODO about slider sync; change tracking layer visuals to use symbol="cross", opacity=0.85, and border_color="green" to better distinguish results. - tracking/ui/worker.py: fix logging call by using self.thread instead of self.thread() to avoid a mistaken call in the thread information. * Update src/napari_deeplabcut/tracking/ui/worker.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/napari_deeplabcut/tracking/core/models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/napari_deeplabcut/_tests/config/test_keybinds.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Improve tracking widgets and lazy torch import Use a lifecycle manager in TrackingControls and tighten viewer typing; update test helper to locate the TrackingControls dock by isinstance and add TYPE_CHECKING napari types. Fix a TYPE_CHECKING import in tracking models (correct TrackingModel import). Replace top-level torch import in the tracking worker with a lazy import and a clear ImportError message advising how to install the tracking extras, and improve CUDA cache cleanup logging. These changes reduce heavy top-level imports, improve typing correctness, and make missing-dependency errors clearer. * Run pre-commit * Tracking UI: Merge tracking results (#190) * Add experimental tracking feature (UI, worker, core) Introduce an experimental point-tracking subsystem: adds a TrackingControls UI, a background TrackingWorker, and core data/models for registering and running trackers. Implements concrete modules under tracking/core (data and abstract TrackingModel with registry), a Qt widget under tracking/_widgets.py, and a worker implementation under tracking/_worker.py. Includes tests (fixtures, widget and worker tests) and a README documenting usage and keybindings; tests register a DummyTracker for unit testing. Adds "torch" to pyproject.toml and updates tox.ini to support the new tracking tests. Co-Authored-By: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com> * Add tracking merge logic for Points layers Introduce src/napari_deeplabcut/tracking/core/merge.py implementing merge preview and apply workflows for merging tracking-result Points into DLC Points layers. Adds data models (TrackingMergePreview, LayerFingerprint, TrackingMergeConflictEntry) and core APIs preview_tracking_merge and apply_tracking_merge with staleness checks via fingerprints, conflict detection, and a FILL_MISSING policy. Includes robust normalization/validation helpers for Napari Points data/features, coordinate-tolerance comparisons, duplicate-slot detection, and safe feature merging that preserves target schema. * Support tracking-result Points layers Add lifecycle manager support for DLC tracking-result Points layers: import the tracking metadata helpers and expose methods to read tracking metadata, classify tracking-result layers, extract source layer names, detect mergeable DLC Points layers, iterate tracking/mergeable layers, and suggest a default merge target. These methods centralize viewer/session-facing semantics for tracking results and provide merge-selection logic (managed vs live, name matching, active layer preference). Also rename the helper in tracking core from is_tracking_result_layer to is_tracking_result_points_layer and update imports accordingly. * Add tracking merge UI and workflow Introduce a new tracking merge UI and workflow to merge tracked points into DLC Points layers. Adds a new module tracking/ui/merger.py implementing TrackingMergeWorkflow, TrackingMergeDialog, and TrackingMergeConflictsDialog with preview, conflict reporting, and application logic (uses preview_tracking_merge and apply_tracking_merge). Integrates a "Merge tracked points…" button into TrackingControls (_widgets.py), wiring it to open the workflow (with a hinted active source), and adds the button to the controls layout. The workflow validates candidates, shows transactional dialogs, applies the merge to the target layer, updates the viewer selection/status, and marks layer presentation changes with error handling and user-facing warnings. * Add tests for tracking merge behaviors Add comprehensive tests for tracking merge utilities: fingerprint_points_layer, preview_tracking_merge, and apply_tracking_merge. Tests cover valid/invalid previews (same layer, duplicate semantic slots), classification of identical/conflict/appendable/invalid rows, preservation of target schema and prevention of source-only column leakage, handling of no-op merges (returns copies), detection of stale previews when layer name or feature columns change, use of id in semantic identity, and coordinate-tolerance effects on identical vs conflict. Also add a small header comment to the tracking conftest. * Restore changes from parent branch * Include preview reason in invalid merge error Improve diagnostics when applying an invalid tracking merge preview by appending preview.invalid_reason to the raised ValueError (defaults to 'Unknown reason.'). This change makes the failure message more informative without altering control flow or behavior. * Escape layer names in merge UI, simplify logs HTML-escape source/target layer display names shown in the tracking merge dialogs to avoid HTML injection and rendering issues (adds html.escape import). Also remove explicit exc_info argument passed to logger.exception calls, relying on logger.exception to include exception info by default. * Validate coordinate columns before merge Add a pre-merge validation that ensures the source (append_df) contains all coordinate columns required by the target layer. If any coord_* columns are missing, the code now raises a ValueError with a clear message listing the missing required columns and the coord_* columns present in the source, and suggests refreshing the preview or merging into a compatible target. This prevents silent dimensionality mismatches and potential data corruption during tracking merges. * Add review-only mode to merge conflicts dialog Introduce a review_only mode to TrackingMergeConflictsDialog so it can be used both as an informational "Review conflicts" viewer and as a confirmation dialog before applying a partial merge. Adds a review_only parameter, adjusts window titles and summary text, and changes the button row (Close for review mode; Cancel/Merge for confirm mode). Exposes a new static review(...) helper and calls it from the preview path. Also simplifies label text construction by using preview.source_layer_name/target_layer_name directly. * Make merge button more distinct * Handle tracking-result points in trajectory plot Integrate the LayerLifecycleManager with the trajectory canvas and improve selection of plottable Points layers. Add manager methods (is_plottable_traj_layer, iter_plottable_traj_layers, suggest_plottable_traj_layer) to determine and suggest eligible layers in viewer order, prioritizing active and managed layers. Have TrajectoryMatplotlibCanvas use get_or_create_layer_manager, refresh on layer insert/remove/selection events, and ask the manager for the plot layer. Add logic to merge tracking-result layer.features and properties into a properties mapping so tracking outputs can be plotted, and use that mapping when building the DataFrame for plotting. Add defensive checks and debug logs to avoid plotting incomplete or incompatible Points layers. * Standardize logger names to module __name__ Replace hard-coded logger strings with logging.getLogger(__name__) in several modules (widgets, layer lifecycle manager and registry) and update tests and conftest to use the underscore package name 'napari_deeplabcut' instead of the hyphen variant. This aligns logger names with module __name__ and keeps logging configuration and test expectations consistent; no functional behavior changes intended. * Add future-delete refine, utils; refactor merge Introduce tracking refine and utility modules and wire a new UI action to delete tracked points in future frames. - Add tracking/core/refine.py: preview_delete_tracking_points_in_future and apply_delete_tracking_points_in_future plus TrackingFutureDeletePreview dataclass to support previewing and applying bulk deletion of matching tracked points in future frames. - Add tracking/core/utils.py: utilities for robust Points layer handling (extract_layer_data_and_features, coord_columns_for_data, pick_semantic_series, coerce_frame_series, normalize_slot_id, slot key helpers, normalize_points_layer_for_tracking) and helpers for generating stable tracked-layer names (make_tracking_iteration_name and related helpers). - Update widgets (_widgets.py): add a "Delete selected points in future frames" button, preview/apply flow with confirmations and notifications, improved error messages, use make_tracking_iteration_name for new tracked layers, and minor UI/log cleanup. - Refactor tracking/core/merge.py to reuse the new utils (coord/normalize/extract/format helpers), remove duplicated helper implementations, and adapt preview/apply logic to new utility APIs. These changes centralize Points data/feature normalization and naming logic, add a user-facing refine action for tracked predictions, and reduce duplication across merge/refine code paths. * Refactor tracking name generation & add typing Add TYPE_CHECKING imports and napari type hints; update spawn.py viewer helpers to use forward-referenced napari.Viewer types (non-functional). Refactor tracking utils to accept an explicit LayerLifecycleManager instead of relying on self, move get_or_create_layer_manager import into the module, and derive a family regex via _tracking_version_family_pattern so tracked-layer versions are incremented per (source, tracker) family regardless of reference frame. Small cleanup and formatting adjustments. * Add deferred keypoint combo selection with QTimer Import QTimer and add _select_keypoint_combo_layer to TrackingControls. The new helper resets the keypoint combo choices and schedules selecting a newly added Points layer on the next event-loop tick to avoid race conditions during layer insertion. Includes robust try/except logging around choice reset and selection, and leaves a FIXME about the unowned QTimer usage. * Update _widgets.py * Update widget_factory.py * Run pre-commit * Add overwrite-existing merge policy Introduce a new TrackingMergePolicy.OVERWRITE_EXISTING and extend the merge preview/apply workflow to support overwriting existing target points. preview_tracking_merge now classifies mismatches as either conflicts (FILL_MISSING) or overwrites (OVERWRITE_EXISTING), tracking counts, indices, entries and truncated counts for both. apply_tracking_merge can now apply overwrite merges by replacing target coordinates and updating shared feature columns, while still supporting appending new points. UI changes: merge dialog now exposes a policy combo, preview summary/details and confirmation dialogs were updated and renamed (TrackingMergeReviewDialog) to show either conflicts or overwrites with adjusted wording and buttons. Default behavior remains fill-missing; validation and stale-preview fingerprint checks preserved. * Allow tracking-result and generic Points as merge targets Generalize merge target logic and update UI text/selection accordingly. - LayerLifecycleManager: refine is_mergeable_dlc_points_layer to: - disallow config-placeholder layers early - validate layer.data shape safely - allow tracking-result layers as merge targets (but they do not count as "managed" when require_managed=True) - clarify docstring and ordering semantics for iter_mergeable_dlc_points_layers - Tracking UI (merger.py): - update user-facing strings to refer to generic "Points" layers instead of "DLC" layers - change target selection logic to exclude the selected source layer from the candidate list - adjust the no-target warning text to reflect the broader set of acceptable Points targets These changes broaden supported merge targets to include tracking-result Points, improve robustness when inspecting layer data, and make the UI wording and selection logic consistent with the new semantics. * Use median for uniform point size and apply it Replace nanmean with nanmedian when computing a uniform point size to avoid outlier-driven values. Exported get_uniform_point_size and use it when copying point sizes to a new layer, with a nested fallback to deepcopy(source.size) to preserve robustness if the uniform computation fails. * Add unit helpers and tests for tracking utils Introduce pure-unit test helpers in conftest (fake_points_layer_factory, dummy_viewer_factory, tracking_manager_factory, patch_tracking_manager) to avoid requiring a real napari viewer or qtbot. Add a new test module tracking/test_utils.py that exercises napari_deeplabcut.tracking.core.utils: extract_layer_data_and_features, coord/semantic helpers, normalize_points_layer_for_tracking, and tracked naming helpers, covering expected behavior and edge cases. * Refactor tracking tests; add refine tests Replace ad-hoc _make_points_layer with a flexible fake_points_layer_factory in tracking tests (conftest) that can produce real napari Points layers or lightweight fake objects and supports explicit features/properties, labels, ids, and extra feature columns. Update test_merge to use the factory, add and adjust assertions for overwrite-aware merge behavior (new overwriteable/overwrites checks and TrackingMergePolicy OVERWRITE_EXISTING tests), and add defensive checks for invalid/stale previews. Add a new test_refine.py containing comprehensive unit tests for preview/apply delete-in-future refine operations. Also add a short header comment to refine.py. These changes improve test coverage and make tests more flexible for both real-layer and pure-unit scenarios. * Reject 'nan'/'none' labels; clean up tests Tighten label validation and tidy test files. - tracking/core/utils.py: Improve normalize_points_layer_for_tracking to treat NaN/None, empty strings, and literal strings 'nan' or 'none' (case-insensitive) as invalid labels by normalizing and explicitly excluding these values. - _tests/tracking/conftest.py: Replace quoted forward-reference type hints with real types for TrackingWorkerData/TrackingWorkerOutput in DummyTracker methods; minor formatting cleanups. - _tests/tracking/test_merge.py and test_refine.py: Whitespace and import reordering, trailing-whitespace removal, and ensure final newline; no behavioral test changes. These changes make label filtering more robust and clean up test code/typing for clarity. * Move tracking widgets tests * Add tracking merge UI tests Add a new test module src/napari_deeplabcut/_tests/tracking/ui/test_merge_ui.py that provides comprehensive unit tests for the tracking merge UI. Tests cover helper utilities (_layer_display_name, preview summary/details), the review dialog, main merge dialog behavior (policies, enablement, summaries), and the workflow (no-candidate warnings, successful merge application, and overwrite confirmation). Uses pytest, qtbot, monkeypatch, and fake_points_layer_factory to simulate layers and preview/apply behaviour. * Create __init__.py * Add tests for tracking data utilities Introduce a new test module covering napari_deeplabcut.tracking.core.data utilities. Tests validate coerce_features_df (returns defensive copy, accepts mappings), add_query_identity_columns (adds tracking columns and preserves originals), expand_query_features_over_time (repeats seed features over frames, sets tracking fields, accepts multiple visibility shapes and errors on mismatch), build_tracking_result_metadata (preserves source metadata, returns deep copy, handles None), and is_tracking_result_points_layer (detects valid tracking metadata and rejects missing/wrong cases). * Add unit tests for tracking UI behaviors Add multiple pytest unit tests covering tracking UI behavior and edge cases. New tests verify: no tracking is requested without a video layer; invalid target frames do not emit requests (parametrized for forward/backward); tracking requests are ignored while already tracking; both-way tracking triggers backward tracking only once (single-shot behavior); tracking_finished creates and selects a result keypoints layer; forward/backward absolute/relative controls and sliders stay in sync; and seed query extraction returns only reference-frame keypoints and raises when none are present. Tests use the qtbot fixture and existing test helpers to set up layers and frames. * Refactor tracking UI tests: fixtures & helpers Split the monolithic setup_tracking_widget fixture into focused fixtures and helpers to improve test clarity and isolation. Added __future__.annotations import and new helpers: patch_tracking_side_effects (disables plugin hooks), empty_tracking_env, tracking_env, _stub_worker_start, _capture_tracking_requests, plus small refactorings to _set_current_tracking_frame and _put_all_points_on_frame. Tests were converted to use the new fixtures, parameterized where appropriate, reorganized into logical sections, and updated to avoid spawning real worker threads or relying on UI insert hooks. Adjusted assertions and timing/wait logic and consolidated result-layer integration test. Overall this makes the tests less brittle and easier to reason about. * Add unit tests for tracking models and Cotracker3 Add comprehensive unit tests for tracking models. The new tests (src/.../_tests/tracking/test_models.py) cover register_backbone, TrackingModel initialization and device/model setup, and many Cotracker3 behaviors: prepare_inputs (xy swap without mutating inputs), prepare_outputs (flattening tracks, restoring plugin xy, handling backward tracking/time reversal), and validate_outputs (acceptance and multiple rejection cases for missing metadata, wrong row counts, shape/time/seed-feature mismatches). Also apply a minor comment tweak in models.py suggesting alternate module layout. * Prevent saving tracking-result Points layers Add pre-save validation and user feedback for Points layers in save workflow. Introduces helper methods to detect tracking-result layers and validate saveable DLC Points (checks type, tracking-result, config-placeholder, and header). Single-layer saves now block direct saving of tracking-result layers with a warning dialog and log entry; non-DLC/invalid Points show a warning. Multi-layer saves filter candidate layers into tracking and saveable sets, persist folder UI state only for saveable DLC layers, call viewer.layers.save for the selectable set, and return a SaveOutcome that notes any skipped tracking-result layers (with guidance to merge them first). * Only route DLC saves for DLC Points layers Make saving behavior more selective: only route a single Points layer through the DLC-specific save path when it is a DeepLabCut-saveable points layer. For tracking-result or foreign/generic Points layers, use the generic napari save instead. Persist folder UI state only for true DLC-saveable layers. Adjust tracking-layer collection and update the post-save status message to reflect that tracking-result layers were saved generically and not as DeepLabCut project data. * remove redundant checks Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Fix debug consistency * Lazy widget imports in factory * Revert "Merge branch 'cy/tracking-merge-results' of https://github.com/DeepLabCut/napari-deeplabcut into cy/tracking-merge-results" This reverts commit 0d93660, reversing changes made to 61955c7. * Allow saving of tracking-result points layers Remove the checks that blocked direct saving of tracking-result Points layers and of non-saveable DeepLabCut Points layers so they can be saved via the generic path. Update the final status message to clarify that tracking-result layers are not saved as DeepLabCut project data and will not be reloaded as proper annotations. The promotion/save target flow remains in place. * Simplify Points post-save handling Use a generic Points check and managed points list for post-save persistence (isinstance(ly, Points) and layer_manager.managed_points_layers()) instead of DLC-specific saveability checks. Remove the special-case handling and warning for tracking-result layers so saves always return a generic success outcome. This simplifies the save workflow and avoids emitting a tracking-specific message. * Update `ClickableLabel` in color scheme reference: don't use colored text (#206) * bugfix: set part_label text color, together with box label * Parts label text: don't set and clear color; only boldfont * Refactor ClickableLabel hover and LabelPair color Introduce a reactive color property and hover-state handling for ClickableLabel: store the pre-hover stylesheet, apply hover color while preserving existing styles (with semicolon handling), and track _hovered state so setting color updates the hover appearance. Call super() in mousePressEvent/enterEvent/leaveEvent to ensure proper event propagation. Add a color property to LabelPair that updates both the color swatch and the nested part label, consolidating previous duplicate logic. --------- Co-authored-by: C-Achard <cyril.achard@epfl.ch> * Tracking fixes (#211) * Sync tracking controls with viewer dims Replace inline lambda with a dedicated slot and add an initial sync so tracking controls reflect the viewer's current dims on startup. Adds _sync_from_viewer_dims (scheduled once after layout) and _on_current_step_changed to robustly update the reference spinbox and frame controls with exception handling and debug logging. When tracking finishes, capture the viewer's current_step and restore it after creating the tracked layer; _select_keypoint_combo_layer signature now accepts an optional restore_step and will set dims.current_step when selecting the layer. * Add themed help/info icons and use in widgets Add a small icon utility and themed SVG assets to standardize help/info buttons across light/dark themes. Introduces src/napari_deeplabcut/ui/icons.py with apply_help_info_icon that picks an appropriate SVG (info_icon_light.svg / info_icon_dark.svg) based on napari appearance settings (with sane fallbacks) and applies icon, size, cursor and tooltip to QToolButton. Replaced ad-hoc QIcon.fromTheme usage in tracking/_widgets.py and layer_stats.py, added two SVG assets, and thread the viewer through KeypointControls -> LayerStatusPanel so the panel can resolve the current theme. This makes help/info buttons consistent across themes and preserves previous sizing/behavior. * Use indeterminate progress for tracking start Don't force the UI progress bar to 0 when tracking starts; set the progress bar maximum to 0 to show an indeterminate/busy state and let the model drive progress updates. Update Cotracker3 to invoke progress_callback(0, 0) at start because it doesn't support intermediate progress updates, treating the start as unknown progress until the model reports completion. * Include SVG info icons in package data Add info_icon_light.svg and info_icon_dark.svg to MANIFEST.in and update pyproject.toml to include assets/*.svg in the napari_deeplabcut package data. This ensures the SVG icons are packaged and distributed with the project so they are available at runtime. * Fix trajectory plot x-axis not updating for tracking results (#210) * Refactor trajectory plotting into plot state Introduce TrajectorySeries and TrajectoryPlotState dataclasses and refactor the trajectory plotting code to build and render an explicit plot state instead of relying on a widget-global DataFrame. Key changes: - Add src/napari_deeplabcut/ui/plots/plot_models.py with immutable dataclasses for series and plot state. - Replace usage of self.df with self._plot_state and store frame bounds and image height in the state. - Add helper methods: _reset_axes, _frame_values_from_layer_data, _image_height_from_viewer, _frame_bounds_from_x, _build_plot_state, and _render_plot_state to centralize logic for deriving x values, image height, and rendering. - Improve robustness and logging when building the plot state; surface a viewer status message on failure. - Simplify and harden color/colormap and plot-mode logic; consolidate plot properties retrieval. - Use frame-space bounds for axis windowing and use draw_idle for non-blocking redraws. - Minor header/comment additions. These changes aim to make plotting more robust, easier to reason about, and fix issues with tracking results plots not being updated accordingly. * Add df property and adjust tests Expose the currently plotted DataFrame via a df property that proxies self._plot_state.df (or None when no plot state exists). Add a guard in _df_has_individuals to handle a missing _plot_state. Update tests to stop assigning canvas.df directly and instead reference the plot_state-based df (commented), keeping test semantics while matching the new internal API. * Fix config segfault from layer removal and overhaul dialog (#208) * Replace placeholder-config decision flow Replace the old 'merge' API with a focused placeholder-config decision flow. Introduces PlaceholderConfigAction and PlaceholderConfigDecisionProvider (replacing MergeDecision* types) and updates __init__ exports. Key changes in LayerLifecycleManager: new PointsInsertResult enum, renamed/rewired merge logic to _maybe_merge_config_points_layer -> returns explicit PlaceholderConfigAction or None, _resolve_placeholder_config_action to consult the new provider, refined handling for APPLY_TO_CURRENT / KEEP_AS_SEPARATE_LAYER / CANCEL (including deferred removal and emitting points_layers_merged_requested only on apply), and adjusted layer insert event emission. UI widget updated to set_placeholder_config_decision_provider and to present a three-button dialog asking how to handle added keypoints. Overall this centralizes and clarifies placeholder config handling and fixes problematic layer removal by delaying the deletion slightly. * Use PlaceholderConfigAction in e2e save test Import PlaceholderConfigAction and set keypoint_controls.resolve_placeholder_config_action to return APPLY_TO_CURRENT in test_save_e2e.py. This forces the placeholder-config resolution during the config merge step to the intended path. * Re-export PlaceholderConfigAction and add types Import PlaceholderConfigAction from the package-level layer_lifecycle module instead of the internal merge submodule, and add explicit type hints to KeypointControls.resolve_placeholder_config_action parameters (Points, tuple[Points, ...], tuple[str, ...], str). Updated the corresponding test import. No functional changes intended—this clarifies the public API and improves static type checking. * Use constant for delay; improve placeholder logic Introduce LAYER_REMOVAL_DELAY_MS and replace hard-coded 300ms timers with the constant to avoid magic numbers. Refactor placeholder config resolution by introducing a _default_action helper (chooses APPLY_TO_CURRENT when no added keypoints, otherwise KEEP_AS_SEPARATE_LAYER). Add robust error handling and logging for the placeholder decision provider: catch exceptions, log debug info, validate the provider return type and warn if invalid, and fall back to the default action. * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Use header equality to decide placeholder action Compare reference and new headers via model_dump() and pass a headers_match flag into the placeholder resolution. Build a clearer message when keypoints were added or when headers differ, and adjust the default action: apply config to the current layer if headers match, otherwise keep as a separate layer. Also rename some local vars for clarity and remove the old one-line message construction. --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * fix trajectories limits; adhere to requested window when close to boundaries (#213) --------- Co-authored-by: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com> * Remove outdated docs See DeepLabCut/DeepLabCut#3318 * Ensure missing torch is properly handled * Tag all co authors Co-Authored-By: Niels <45132115+n-poulsen@users.noreply.github.com> Co-Authored-By: Bryan Gotti <29003872+brygotti@users.noreply.github.com> Co-Authored-By: Jun Huang <44811884+junhuang7@users.noreply.github.com> Co-Authored-By: Riccardo Carpineto <92370991+riccardoprog@users.noreply.github.com> Co-Authored-By: maud73 <92534343+maud73@users.noreply.github.com> Co-Authored-By: Alexis Cogne <133367958+alexiscogne@users.noreply.github.com> Co-Authored-By: Jennifer Ayer <80843866+antigonej@users.noreply.github.com> Co-Authored-By: Laura Gambaretto <92331544+lauragambaretto@users.noreply.github.com> Co-Authored-By: Lucas Stoffl <37299767+luczot@users.noreply.github.com> Co-Authored-By: Aykelia <114958332+aykelia@users.noreply.github.com> --------- Co-authored-by: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com> Co-authored-by: Niels <45132115+n-poulsen@users.noreply.github.com> Co-authored-by: Bryan Gotti <29003872+brygotti@users.noreply.github.com> Co-authored-by: Jun Huang <44811884+junhuang7@users.noreply.github.com> Co-authored-by: Riccardo Carpineto <92370991+riccardoprog@users.noreply.github.com> Co-authored-by: maud73 <92534343+maud73@users.noreply.github.com> Co-authored-by: Alexis Cogne <133367958+alexiscogne@users.noreply.github.com> Co-authored-by: Jennifer Ayer <80843866+antigonej@users.noreply.github.com> Co-authored-by: Laura Gambaretto <92331544+lauragambaretto@users.noreply.github.com> Co-authored-by: Lucas Stoffl <37299767+luczot@users.noreply.github.com> Co-authored-by: Aykelia <114958332+aykelia@users.noreply.github.com>
C-Achard
added a commit
to DeepLabCut/napari-deeplabcut
that referenced
this pull request
May 18, 2026
* Create v0_3_1_0.md * Tracking: rebased and refactored Cotracker integration (#187) * Add experimental tracking feature (UI, worker, core) Introduce an experimental point-tracking subsystem: adds a TrackingControls UI, a background TrackingWorker, and core data/models for registering and running trackers. Implements concrete modules under tracking/core (data and abstract TrackingModel with registry), a Qt widget under tracking/_widgets.py, and a worker implementation under tracking/_worker.py. Includes tests (fixtures, widget and worker tests) and a README documenting usage and keybindings; tests register a DummyTracker for unit testing. Adds "torch" to pyproject.toml and updates tox.ini to support the new tracking tests. Co-Authored-By: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com> * Restore plugin tracking manifest Co-Authored-By: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com> * Make tracking shortcuts configurable & conditional Introduce configurable tracking shortcuts and only enable them when the tracking widget is visible. Add TRACKING_SHORTCUTS_ENABLED (env NAPARI_DLC_TRACKING_SHORTCUTS_ENABLED, default enabled) to settings, centralize tracking key definitions in config/keybinds.py, and have iter_shortcuts yield tracking shortcuts only when enabled. Update TrackingControls to import those key configs, set an objectName/property for dock detection, and guard bound key callbacks so they only act when the tracking dock is open/visible. Enhance Shortcuts dialog to recognize the new "tracking-points-layer" scope and to hide/disable shortcuts when the tracking widget isn't open. Also update imports (KeypointStore path) and remove duplicate dataclass/keybind definitions. * Fix shortcut tests * Improve tracking feature pipeline and result layers Introduce an identity-preserving tracking data pipeline and create separate tracking result layers. data.py: add tracking metadata constants, convert TrackingWorkerData/Output to use DataFrame for per-query features, and add helpers (coerce_features_df, add_query_identity_columns, expand_query_features_over_time, build_tracking_result_metadata, is_tracking_result_layer) to preserve query identity and expand features over time. models.py: update Cotracker3 to avoid mutating inputs, correctly handle x/y ordering, visibility shapes, time-reversal for backward tracking, and produce flattened keypoints plus expanded per-point feature rows. _widgets.py: add UI helpers to seed query points/features, build a non-destructive tracking result Points layer (with metadata and visual tweaks), wire those into the tracking flow, improve error handling, and select the created layer in the viewer. Overall these changes ensure semantic identity of query points is preserved through inference and that results are stored in a dedicated, annotated layer. * Fix tracking tests * Fix incorrect dataclass use * Improve tracking tests: seed frames and assertions Add helpers to place all test keypoints on a specific frame and to drive the widget reference frame (_put_all_points_on_frame, _set_current_tracking_frame). Update tests (test_backward_track, test_bothway_track) to use these helpers instead of directly mutating viewer.dims, and add assertions verifying the tracking request payload: correct reference_frame_index, sliced video length, keypoints rebased to local frame 0, original keypoint feature columns (id/name) preserved, and new tracking identity columns (tracking_query_index, tracking_query_frame) populated. Also ensure both-way tracking uses the same seed frame and that forward==reference only triggers backward tracking. * Remove duplicate worker * Use queued signals and refine TrackingWorker signals Switches several signal/slot connections to explicit Qt.QueuedConnection to ensure safe cross-thread delivery and replaces tuple/object-typed signals with primitive signatures. In TrackingControls: trackingRequested now uses Signal(object); worker connections use queued delivery; added _on_worker_started/_on_worker_finished/_on_worker_progress slots, a _request_worker_stop wrapper, and a _debug_thread helper for logging. In TrackingWorker: progress is now Signal(int,int), trackingFinished is Signal(object), track is annotated with Slot(object), and progress.emit now emits two ints; added thread debug logging. These changes improve thread-safety, clarity of signal payloads, and progress reporting. * Refactor TrackingWorker stop flow and logging Update tracking worker and UI to improve stop handling, logging, and error reporting. - TrackingWorker: replace boolean stop flag with threading.Event, add request_stop(), use self.thread for logging, emit trackingStarted/trackingStopped/trackingFinished and finished in appropriate places, avoid QCoreApplication.processEvents, and guard torch.cuda.empty_cache. Ensure early return when stop requested. - Tests: adapt progress signal handling to accept (current, total) args. - Widgets: use self.thread attribute in debug logs, derive reference frame index from returned features when available, catch ValueError from seed point queries and show a napari warning instead of crashing, plus small formatting fixes. These changes make worker shutdown safer across threads, improve diagnostics, and prevent main-thread operations from being performed in the worker. * Use CoTracker offline model and fix visibility Switch the CoTracker backend to the offline model and refactor run() to perform whole-clip inference. Inputs.video is converted and permuted to (1, T, C, H, W), queries are batched to (1, K, 3), inference is executed under torch.inference_mode, progress callbacks updated, and a RuntimeError is raised if no predictions are returned. Also add early-stop handling (return None). Additionally, fix visibility array handling in expand_query_features_over_time by squeezing a leading singleton channel dimension when vis.ndim == 3 and vis.shape[0] == 1. * Adjust tracking UI visuals and threading bugfix Tweak tracking UI appearance and fix a threading/logging bug. - tracking/_widgets.py: add TODO about slider sync; change tracking layer visuals to use symbol="cross", opacity=0.85, and border_color="green" to better distinguish results. - tracking/ui/worker.py: fix logging call by using self.thread instead of self.thread() to avoid a mistaken call in the thread information. * Update src/napari_deeplabcut/tracking/ui/worker.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/napari_deeplabcut/tracking/core/models.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update src/napari_deeplabcut/_tests/config/test_keybinds.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Improve tracking widgets and lazy torch import Use a lifecycle manager in TrackingControls and tighten viewer typing; update test helper to locate the TrackingControls dock by isinstance and add TYPE_CHECKING napari types. Fix a TYPE_CHECKING import in tracking models (correct TrackingModel import). Replace top-level torch import in the tracking worker with a lazy import and a clear ImportError message advising how to install the tracking extras, and improve CUDA cache cleanup logging. These changes reduce heavy top-level imports, improve typing correctness, and make missing-dependency errors clearer. * Run pre-commit * Tracking UI: Merge tracking results (#190) * Add experimental tracking feature (UI, worker, core) Introduce an experimental point-tracking subsystem: adds a TrackingControls UI, a background TrackingWorker, and core data/models for registering and running trackers. Implements concrete modules under tracking/core (data and abstract TrackingModel with registry), a Qt widget under tracking/_widgets.py, and a worker implementation under tracking/_worker.py. Includes tests (fixtures, widget and worker tests) and a README documenting usage and keybindings; tests register a DummyTracker for unit testing. Adds "torch" to pyproject.toml and updates tox.ini to support the new tracking tests. Co-Authored-By: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com> * Add tracking merge logic for Points layers Introduce src/napari_deeplabcut/tracking/core/merge.py implementing merge preview and apply workflows for merging tracking-result Points into DLC Points layers. Adds data models (TrackingMergePreview, LayerFingerprint, TrackingMergeConflictEntry) and core APIs preview_tracking_merge and apply_tracking_merge with staleness checks via fingerprints, conflict detection, and a FILL_MISSING policy. Includes robust normalization/validation helpers for Napari Points data/features, coordinate-tolerance comparisons, duplicate-slot detection, and safe feature merging that preserves target schema. * Support tracking-result Points layers Add lifecycle manager support for DLC tracking-result Points layers: import the tracking metadata helpers and expose methods to read tracking metadata, classify tracking-result layers, extract source layer names, detect mergeable DLC Points layers, iterate tracking/mergeable layers, and suggest a default merge target. These methods centralize viewer/session-facing semantics for tracking results and provide merge-selection logic (managed vs live, name matching, active layer preference). Also rename the helper in tracking core from is_tracking_result_layer to is_tracking_result_points_layer and update imports accordingly. * Add tracking merge UI and workflow Introduce a new tracking merge UI and workflow to merge tracked points into DLC Points layers. Adds a new module tracking/ui/merger.py implementing TrackingMergeWorkflow, TrackingMergeDialog, and TrackingMergeConflictsDialog with preview, conflict reporting, and application logic (uses preview_tracking_merge and apply_tracking_merge). Integrates a "Merge tracked points…" button into TrackingControls (_widgets.py), wiring it to open the workflow (with a hinted active source), and adds the button to the controls layout. The workflow validates candidates, shows transactional dialogs, applies the merge to the target layer, updates the viewer selection/status, and marks layer presentation changes with error handling and user-facing warnings. * Add tests for tracking merge behaviors Add comprehensive tests for tracking merge utilities: fingerprint_points_layer, preview_tracking_merge, and apply_tracking_merge. Tests cover valid/invalid previews (same layer, duplicate semantic slots), classification of identical/conflict/appendable/invalid rows, preservation of target schema and prevention of source-only column leakage, handling of no-op merges (returns copies), detection of stale previews when layer name or feature columns change, use of id in semantic identity, and coordinate-tolerance effects on identical vs conflict. Also add a small header comment to the tracking conftest. * Restore changes from parent branch * Include preview reason in invalid merge error Improve diagnostics when applying an invalid tracking merge preview by appending preview.invalid_reason to the raised ValueError (defaults to 'Unknown reason.'). This change makes the failure message more informative without altering control flow or behavior. * Escape layer names in merge UI, simplify logs HTML-escape source/target layer display names shown in the tracking merge dialogs to avoid HTML injection and rendering issues (adds html.escape import). Also remove explicit exc_info argument passed to logger.exception calls, relying on logger.exception to include exception info by default. * Validate coordinate columns before merge Add a pre-merge validation that ensures the source (append_df) contains all coordinate columns required by the target layer. If any coord_* columns are missing, the code now raises a ValueError with a clear message listing the missing required columns and the coord_* columns present in the source, and suggests refreshing the preview or merging into a compatible target. This prevents silent dimensionality mismatches and potential data corruption during tracking merges. * Add review-only mode to merge conflicts dialog Introduce a review_only mode to TrackingMergeConflictsDialog so it can be used both as an informational "Review conflicts" viewer and as a confirmation dialog before applying a partial merge. Adds a review_only parameter, adjusts window titles and summary text, and changes the button row (Close for review mode; Cancel/Merge for confirm mode). Exposes a new static review(...) helper and calls it from the preview path. Also simplifies label text construction by using preview.source_layer_name/target_layer_name directly. * Make merge button more distinct * Handle tracking-result points in trajectory plot Integrate the LayerLifecycleManager with the trajectory canvas and improve selection of plottable Points layers. Add manager methods (is_plottable_traj_layer, iter_plottable_traj_layers, suggest_plottable_traj_layer) to determine and suggest eligible layers in viewer order, prioritizing active and managed layers. Have TrajectoryMatplotlibCanvas use get_or_create_layer_manager, refresh on layer insert/remove/selection events, and ask the manager for the plot layer. Add logic to merge tracking-result layer.features and properties into a properties mapping so tracking outputs can be plotted, and use that mapping when building the DataFrame for plotting. Add defensive checks and debug logs to avoid plotting incomplete or incompatible Points layers. * Standardize logger names to module __name__ Replace hard-coded logger strings with logging.getLogger(__name__) in several modules (widgets, layer lifecycle manager and registry) and update tests and conftest to use the underscore package name 'napari_deeplabcut' instead of the hyphen variant. This aligns logger names with module __name__ and keeps logging configuration and test expectations consistent; no functional behavior changes intended. * Add future-delete refine, utils; refactor merge Introduce tracking refine and utility modules and wire a new UI action to delete tracked points in future frames. - Add tracking/core/refine.py: preview_delete_tracking_points_in_future and apply_delete_tracking_points_in_future plus TrackingFutureDeletePreview dataclass to support previewing and applying bulk deletion of matching tracked points in future frames. - Add tracking/core/utils.py: utilities for robust Points layer handling (extract_layer_data_and_features, coord_columns_for_data, pick_semantic_series, coerce_frame_series, normalize_slot_id, slot key helpers, normalize_points_layer_for_tracking) and helpers for generating stable tracked-layer names (make_tracking_iteration_name and related helpers). - Update widgets (_widgets.py): add a "Delete selected points in future frames" button, preview/apply flow with confirmations and notifications, improved error messages, use make_tracking_iteration_name for new tracked layers, and minor UI/log cleanup. - Refactor tracking/core/merge.py to reuse the new utils (coord/normalize/extract/format helpers), remove duplicated helper implementations, and adapt preview/apply logic to new utility APIs. These changes centralize Points data/feature normalization and naming logic, add a user-facing refine action for tracked predictions, and reduce duplication across merge/refine code paths. * Refactor tracking name generation & add typing Add TYPE_CHECKING imports and napari type hints; update spawn.py viewer helpers to use forward-referenced napari.Viewer types (non-functional). Refactor tracking utils to accept an explicit LayerLifecycleManager instead of relying on self, move get_or_create_layer_manager import into the module, and derive a family regex via _tracking_version_family_pattern so tracked-layer versions are incremented per (source, tracker) family regardless of reference frame. Small cleanup and formatting adjustments. * Add deferred keypoint combo selection with QTimer Import QTimer and add _select_keypoint_combo_layer to TrackingControls. The new helper resets the keypoint combo choices and schedules selecting a newly added Points layer on the next event-loop tick to avoid race conditions during layer insertion. Includes robust try/except logging around choice reset and selection, and leaves a FIXME about the unowned QTimer usage. * Update _widgets.py * Update widget_factory.py * Run pre-commit * Add overwrite-existing merge policy Introduce a new TrackingMergePolicy.OVERWRITE_EXISTING and extend the merge preview/apply workflow to support overwriting existing target points. preview_tracking_merge now classifies mismatches as either conflicts (FILL_MISSING) or overwrites (OVERWRITE_EXISTING), tracking counts, indices, entries and truncated counts for both. apply_tracking_merge can now apply overwrite merges by replacing target coordinates and updating shared feature columns, while still supporting appending new points. UI changes: merge dialog now exposes a policy combo, preview summary/details and confirmation dialogs were updated and renamed (TrackingMergeReviewDialog) to show either conflicts or overwrites with adjusted wording and buttons. Default behavior remains fill-missing; validation and stale-preview fingerprint checks preserved. * Allow tracking-result and generic Points as merge targets Generalize merge target logic and update UI text/selection accordingly. - LayerLifecycleManager: refine is_mergeable_dlc_points_layer to: - disallow config-placeholder layers early - validate layer.data shape safely - allow tracking-result layers as merge targets (but they do not count as "managed" when require_managed=True) - clarify docstring and ordering semantics for iter_mergeable_dlc_points_layers - Tracking UI (merger.py): - update user-facing strings to refer to generic "Points" layers instead of "DLC" layers - change target selection logic to exclude the selected source layer from the candidate list - adjust the no-target warning text to reflect the broader set of acceptable Points targets These changes broaden supported merge targets to include tracking-result Points, improve robustness when inspecting layer data, and make the UI wording and selection logic consistent with the new semantics. * Use median for uniform point size and apply it Replace nanmean with nanmedian when computing a uniform point size to avoid outlier-driven values. Exported get_uniform_point_size and use it when copying point sizes to a new layer, with a nested fallback to deepcopy(source.size) to preserve robustness if the uniform computation fails. * Add unit helpers and tests for tracking utils Introduce pure-unit test helpers in conftest (fake_points_layer_factory, dummy_viewer_factory, tracking_manager_factory, patch_tracking_manager) to avoid requiring a real napari viewer or qtbot. Add a new test module tracking/test_utils.py that exercises napari_deeplabcut.tracking.core.utils: extract_layer_data_and_features, coord/semantic helpers, normalize_points_layer_for_tracking, and tracked naming helpers, covering expected behavior and edge cases. * Refactor tracking tests; add refine tests Replace ad-hoc _make_points_layer with a flexible fake_points_layer_factory in tracking tests (conftest) that can produce real napari Points layers or lightweight fake objects and supports explicit features/properties, labels, ids, and extra feature columns. Update test_merge to use the factory, add and adjust assertions for overwrite-aware merge behavior (new overwriteable/overwrites checks and TrackingMergePolicy OVERWRITE_EXISTING tests), and add defensive checks for invalid/stale previews. Add a new test_refine.py containing comprehensive unit tests for preview/apply delete-in-future refine operations. Also add a short header comment to refine.py. These changes improve test coverage and make tests more flexible for both real-layer and pure-unit scenarios. * Reject 'nan'/'none' labels; clean up tests Tighten label validation and tidy test files. - tracking/core/utils.py: Improve normalize_points_layer_for_tracking to treat NaN/None, empty strings, and literal strings 'nan' or 'none' (case-insensitive) as invalid labels by normalizing and explicitly excluding these values. - _tests/tracking/conftest.py: Replace quoted forward-reference type hints with real types for TrackingWorkerData/TrackingWorkerOutput in DummyTracker methods; minor formatting cleanups. - _tests/tracking/test_merge.py and test_refine.py: Whitespace and import reordering, trailing-whitespace removal, and ensure final newline; no behavioral test changes. These changes make label filtering more robust and clean up test code/typing for clarity. * Move tracking widgets tests * Add tracking merge UI tests Add a new test module src/napari_deeplabcut/_tests/tracking/ui/test_merge_ui.py that provides comprehensive unit tests for the tracking merge UI. Tests cover helper utilities (_layer_display_name, preview summary/details), the review dialog, main merge dialog behavior (policies, enablement, summaries), and the workflow (no-candidate warnings, successful merge application, and overwrite confirmation). Uses pytest, qtbot, monkeypatch, and fake_points_layer_factory to simulate layers and preview/apply behaviour. * Create __init__.py * Add tests for tracking data utilities Introduce a new test module covering napari_deeplabcut.tracking.core.data utilities. Tests validate coerce_features_df (returns defensive copy, accepts mappings), add_query_identity_columns (adds tracking columns and preserves originals), expand_query_features_over_time (repeats seed features over frames, sets tracking fields, accepts multiple visibility shapes and errors on mismatch), build_tracking_result_metadata (preserves source metadata, returns deep copy, handles None), and is_tracking_result_points_layer (detects valid tracking metadata and rejects missing/wrong cases). * Add unit tests for tracking UI behaviors Add multiple pytest unit tests covering tracking UI behavior and edge cases. New tests verify: no tracking is requested without a video layer; invalid target frames do not emit requests (parametrized for forward/backward); tracking requests are ignored while already tracking; both-way tracking triggers backward tracking only once (single-shot behavior); tracking_finished creates and selects a result keypoints layer; forward/backward absolute/relative controls and sliders stay in sync; and seed query extraction returns only reference-frame keypoints and raises when none are present. Tests use the qtbot fixture and existing test helpers to set up layers and frames. * Refactor tracking UI tests: fixtures & helpers Split the monolithic setup_tracking_widget fixture into focused fixtures and helpers to improve test clarity and isolation. Added __future__.annotations import and new helpers: patch_tracking_side_effects (disables plugin hooks), empty_tracking_env, tracking_env, _stub_worker_start, _capture_tracking_requests, plus small refactorings to _set_current_tracking_frame and _put_all_points_on_frame. Tests were converted to use the new fixtures, parameterized where appropriate, reorganized into logical sections, and updated to avoid spawning real worker threads or relying on UI insert hooks. Adjusted assertions and timing/wait logic and consolidated result-layer integration test. Overall this makes the tests less brittle and easier to reason about. * Add unit tests for tracking models and Cotracker3 Add comprehensive unit tests for tracking models. The new tests (src/.../_tests/tracking/test_models.py) cover register_backbone, TrackingModel initialization and device/model setup, and many Cotracker3 behaviors: prepare_inputs (xy swap without mutating inputs), prepare_outputs (flattening tracks, restoring plugin xy, handling backward tracking/time reversal), and validate_outputs (acceptance and multiple rejection cases for missing metadata, wrong row counts, shape/time/seed-feature mismatches). Also apply a minor comment tweak in models.py suggesting alternate module layout. * Prevent saving tracking-result Points layers Add pre-save validation and user feedback for Points layers in save workflow. Introduces helper methods to detect tracking-result layers and validate saveable DLC Points (checks type, tracking-result, config-placeholder, and header). Single-layer saves now block direct saving of tracking-result layers with a warning dialog and log entry; non-DLC/invalid Points show a warning. Multi-layer saves filter candidate layers into tracking and saveable sets, persist folder UI state only for saveable DLC layers, call viewer.layers.save for the selectable set, and return a SaveOutcome that notes any skipped tracking-result layers (with guidance to merge them first). * Only route DLC saves for DLC Points layers Make saving behavior more selective: only route a single Points layer through the DLC-specific save path when it is a DeepLabCut-saveable points layer. For tracking-result or foreign/generic Points layers, use the generic napari save instead. Persist folder UI state only for true DLC-saveable layers. Adjust tracking-layer collection and update the post-save status message to reflect that tracking-result layers were saved generically and not as DeepLabCut project data. * remove redundant checks Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Fix debug consistency * Lazy widget imports in factory * Revert "Merge branch 'cy/tracking-merge-results' of https://github.com/DeepLabCut/napari-deeplabcut into cy/tracking-merge-results" This reverts commit 0d93660, reversing changes made to 61955c7. * Allow saving of tracking-result points layers Remove the checks that blocked direct saving of tracking-result Points layers and of non-saveable DeepLabCut Points layers so they can be saved via the generic path. Update the final status message to clarify that tracking-result layers are not saved as DeepLabCut project data and will not be reloaded as proper annotations. The promotion/save target flow remains in place. * Simplify Points post-save handling Use a generic Points check and managed points list for post-save persistence (isinstance(ly, Points) and layer_manager.managed_points_layers()) instead of DLC-specific saveability checks. Remove the special-case handling and warning for tracking-result layers so saves always return a generic success outcome. This simplifies the save workflow and avoids emitting a tracking-specific message. * Update `ClickableLabel` in color scheme reference: don't use colored text (#206) * bugfix: set part_label text color, together with box label * Parts label text: don't set and clear color; only boldfont * Refactor ClickableLabel hover and LabelPair color Introduce a reactive color property and hover-state handling for ClickableLabel: store the pre-hover stylesheet, apply hover color while preserving existing styles (with semicolon handling), and track _hovered state so setting color updates the hover appearance. Call super() in mousePressEvent/enterEvent/leaveEvent to ensure proper event propagation. Add a color property to LabelPair that updates both the color swatch and the nested part label, consolidating previous duplicate logic. --------- Co-authored-by: C-Achard <cyril.achard@epfl.ch> * Tracking fixes (#211) * Sync tracking controls with viewer dims Replace inline lambda with a dedicated slot and add an initial sync so tracking controls reflect the viewer's current dims on startup. Adds _sync_from_viewer_dims (scheduled once after layout) and _on_current_step_changed to robustly update the reference spinbox and frame controls with exception handling and debug logging. When tracking finishes, capture the viewer's current_step and restore it after creating the tracked layer; _select_keypoint_combo_layer signature now accepts an optional restore_step and will set dims.current_step when selecting the layer. * Add themed help/info icons and use in widgets Add a small icon utility and themed SVG assets to standardize help/info buttons across light/dark themes. Introduces src/napari_deeplabcut/ui/icons.py with apply_help_info_icon that picks an appropriate SVG (info_icon_light.svg / info_icon_dark.svg) based on napari appearance settings (with sane fallbacks) and applies icon, size, cursor and tooltip to QToolButton. Replaced ad-hoc QIcon.fromTheme usage in tracking/_widgets.py and layer_stats.py, added two SVG assets, and thread the viewer through KeypointControls -> LayerStatusPanel so the panel can resolve the current theme. This makes help/info buttons consistent across themes and preserves previous sizing/behavior. * Use indeterminate progress for tracking start Don't force the UI progress bar to 0 when tracking starts; set the progress bar maximum to 0 to show an indeterminate/busy state and let the model drive progress updates. Update Cotracker3 to invoke progress_callback(0, 0) at start because it doesn't support intermediate progress updates, treating the start as unknown progress until the model reports completion. * Include SVG info icons in package data Add info_icon_light.svg and info_icon_dark.svg to MANIFEST.in and update pyproject.toml to include assets/*.svg in the napari_deeplabcut package data. This ensures the SVG icons are packaged and distributed with the project so they are available at runtime. * Fix trajectory plot x-axis not updating for tracking results (#210) * Refactor trajectory plotting into plot state Introduce TrajectorySeries and TrajectoryPlotState dataclasses and refactor the trajectory plotting code to build and render an explicit plot state instead of relying on a widget-global DataFrame. Key changes: - Add src/napari_deeplabcut/ui/plots/plot_models.py with immutable dataclasses for series and plot state. - Replace usage of self.df with self._plot_state and store frame bounds and image height in the state. - Add helper methods: _reset_axes, _frame_values_from_layer_data, _image_height_from_viewer, _frame_bounds_from_x, _build_plot_state, and _render_plot_state to centralize logic for deriving x values, image height, and rendering. - Improve robustness and logging when building the plot state; surface a viewer status message on failure. - Simplify and harden color/colormap and plot-mode logic; consolidate plot properties retrieval. - Use frame-space bounds for axis windowing and use draw_idle for non-blocking redraws. - Minor header/comment additions. These changes aim to make plotting more robust, easier to reason about, and fix issues with tracking results plots not being updated accordingly. * Add df property and adjust tests Expose the currently plotted DataFrame via a df property that proxies self._plot_state.df (or None when no plot state exists). Add a guard in _df_has_individuals to handle a missing _plot_state. Update tests to stop assigning canvas.df directly and instead reference the plot_state-based df (commented), keeping test semantics while matching the new internal API. * Fix config segfault from layer removal and overhaul dialog (#208) * Replace placeholder-config decision flow Replace the old 'merge' API with a focused placeholder-config decision flow. Introduces PlaceholderConfigAction and PlaceholderConfigDecisionProvider (replacing MergeDecision* types) and updates __init__ exports. Key changes in LayerLifecycleManager: new PointsInsertResult enum, renamed/rewired merge logic to _maybe_merge_config_points_layer -> returns explicit PlaceholderConfigAction or None, _resolve_placeholder_config_action to consult the new provider, refined handling for APPLY_TO_CURRENT / KEEP_AS_SEPARATE_LAYER / CANCEL (including deferred removal and emitting points_layers_merged_requested only on apply), and adjusted layer insert event emission. UI widget updated to set_placeholder_config_decision_provider and to present a three-button dialog asking how to handle added keypoints. Overall this centralizes and clarifies placeholder config handling and fixes problematic layer removal by delaying the deletion slightly. * Use PlaceholderConfigAction in e2e save test Import PlaceholderConfigAction and set keypoint_controls.resolve_placeholder_config_action to return APPLY_TO_CURRENT in test_save_e2e.py. This forces the placeholder-config resolution during the config merge step to the intended path. * Re-export PlaceholderConfigAction and add types Import PlaceholderConfigAction from the package-level layer_lifecycle module instead of the internal merge submodule, and add explicit type hints to KeypointControls.resolve_placeholder_config_action parameters (Points, tuple[Points, ...], tuple[str, ...], str). Updated the corresponding test import. No functional changes intended—this clarifies the public API and improves static type checking. * Use constant for delay; improve placeholder logic Introduce LAYER_REMOVAL_DELAY_MS and replace hard-coded 300ms timers with the constant to avoid magic numbers. Refactor placeholder config resolution by introducing a _default_action helper (chooses APPLY_TO_CURRENT when no added keypoints, otherwise KEEP_AS_SEPARATE_LAYER). Add robust error handling and logging for the placeholder decision provider: catch exceptions, log debug info, validate the provider return type and warn if invalid, and fall back to the default action. * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Use header equality to decide placeholder action Compare reference and new headers via model_dump() and pass a headers_match flag into the placeholder resolution. Build a clearer message when keypoints were added or when headers differ, and adjust the default action: apply config to the current layer if headers match, otherwise keep as a separate layer. Also rename some local vars for clarity and remove the old one-line message construction. --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * fix trajectories limits; adhere to requested window when close to boundaries (#213) --------- Co-authored-by: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com> * Remove outdated docs See DeepLabCut/DeepLabCut#3318 * Ensure missing torch is properly handled * Tag all co authors Co-Authored-By: Niels <45132115+n-poulsen@users.noreply.github.com> Co-Authored-By: Bryan Gotti <29003872+brygotti@users.noreply.github.com> Co-Authored-By: Jun Huang <44811884+junhuang7@users.noreply.github.com> Co-Authored-By: Riccardo Carpineto <92370991+riccardoprog@users.noreply.github.com> Co-Authored-By: maud73 <92534343+maud73@users.noreply.github.com> Co-Authored-By: Alexis Cogne <133367958+alexiscogne@users.noreply.github.com> Co-Authored-By: Jennifer Ayer <80843866+antigonej@users.noreply.github.com> Co-Authored-By: Laura Gambaretto <92331544+lauragambaretto@users.noreply.github.com> Co-Authored-By: Lucas Stoffl <37299767+luczot@users.noreply.github.com> Co-Authored-By: Aykelia <114958332+aykelia@users.noreply.github.com> --------- Co-authored-by: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com> Co-authored-by: Niels <45132115+n-poulsen@users.noreply.github.com> Co-authored-by: Bryan Gotti <29003872+brygotti@users.noreply.github.com> Co-authored-by: Jun Huang <44811884+junhuang7@users.noreply.github.com> Co-authored-by: Riccardo Carpineto <92370991+riccardoprog@users.noreply.github.com> Co-authored-by: maud73 <92534343+maud73@users.noreply.github.com> Co-authored-by: Alexis Cogne <133367958+alexiscogne@users.noreply.github.com> Co-authored-by: Jennifer Ayer <80843866+antigonej@users.noreply.github.com> Co-authored-by: Laura Gambaretto <92331544+lauragambaretto@users.noreply.github.com> Co-authored-by: Lucas Stoffl <37299767+luczot@users.noreply.github.com> Co-authored-by: Aykelia <114958332+aykelia@users.noreply.github.com> --------- Co-authored-by: Arash Sal Moslehian <57039957+arashsm79@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com> Co-authored-by: Niels <45132115+n-poulsen@users.noreply.github.com> Co-authored-by: Bryan Gotti <29003872+brygotti@users.noreply.github.com> Co-authored-by: Jun Huang <44811884+junhuang7@users.noreply.github.com> Co-authored-by: Riccardo Carpineto <92370991+riccardoprog@users.noreply.github.com> Co-authored-by: maud73 <92534343+maud73@users.noreply.github.com> Co-authored-by: Alexis Cogne <133367958+alexiscogne@users.noreply.github.com> Co-authored-by: Jennifer Ayer <80843866+antigonej@users.noreply.github.com> Co-authored-by: Laura Gambaretto <92331544+lauragambaretto@users.noreply.github.com> Co-authored-by: Lucas Stoffl <37299767+luczot@users.noreply.github.com> Co-authored-by: Aykelia <114958332+aykelia@users.noreply.github.com>
deruyter92
approved these changes
May 19, 2026
Collaborator
Author
|
napari CoTracker is live, would be great to release this whenever we can! |
Update napari documentation to improve clarity and guidance for tracking and labeling workflows. Changes include: - Add a section anchor for the DLC basic workflow in docs/gui/napari/basic_usage.md. - Expand docs/gui/napari/tracking/basic_usage.md to cover loading extracted frames, reference the DLC basic workflow, and add a note about PyTorch/GPU requirements for the [tracking] extra. - Clarify how tracking is initialized (only from the selected reference frame), background behavior, shortcut visibility, and visual styling/accessibility notes. - Add overwrite warnings, better explanation of deleting future tracked points, and explicit guidance on merging and saving (tracking layers are intermediate; tracking layers saved as CSVs are not written back to CollectedData_*.h5). - Recommend running tracking on original videos for continuity and adjust several wording/grammar and link improvements throughout the tracking docs. Files changed: docs/gui/napari/basic_usage.md, docs/gui/napari/tracking/basic_usage.md.
MMathisLab
approved these changes
May 21, 2026
16 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds documentation for the new automated point tracking feature in the napari plugin.
The main focus is on guiding users through the new tracking workflow, requirements, and troubleshooting, as well as providing proper attribution and empirical limitations for the tracking models.
New documentation for automated point tracking:
docs/gui/napari/tracking/basic_usage.md, with detailed instructions and workflow for using the tracking feature in the napari plugin, including requirements, UI walkthrough, keyboard shortcuts, result interpretation, and troubleshooting._toc.yml) to include the new tracking documentation under the napari section.Required