Skip to content

Prepare migration to pandas 3.0 - #3360

Merged
MMathisLab merged 12 commits into
mainfrom
jaap/migrate-pd-3
Jun 25, 2026
Merged

Prepare migration to pandas 3.0#3360
MMathisLab merged 12 commits into
mainfrom
jaap/migrate-pd-3

Conversation

@deruyter92

@deruyter92 deruyter92 commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Pandas released version 3.0 in Jan 2026 which introduces breaking changes for our codebase. It would be useful to address the places where it currently breaks, so we can remove the upperbound version we introduced in #3186. See the full tracking issue #3362, for an overview of related PRs.

Small fixes and flagged patterns:
As exploratory preparation, this PR implements small fixes in preparation to 3.0 migration. Also it flags breaking instances based on a targeted search for these patterns:

  • Chained .loc assignment (copy on write; CoW) -> mutate parent object directly instead of a subset view
  • groupby(..., axis=1) -> groupby on the transposed df
  • Column-level groupby(level=...) -> groupby on the transposed df
  • select_dtypes(include="object") -> string type migration
  • deprecated options like future.no_silent_downcasting

Future mode testing:
Also this PR adds a new tool for CI or local testing, using the pandas 2.3 future options that warns for CoW patterns and detects where str type should be included instead of only object.

@C-Achard C-Achard changed the title migrate to pandas 3.0 Migrate to pandas 3.0 Jun 9, 2026
@C-Achard C-Achard added enhancement New feature or request backwards compatibility issues concerning prior to current versions dependencies Pull requests that update a dependency file labels Jun 9, 2026
@deruyter92 deruyter92 mentioned this pull request Jun 10, 2026
3 tasks
@deruyter92 deruyter92 changed the title Migrate to pandas 3.0 Prepare migration to pandas 3.0 Jun 10, 2026
@deruyter92

deruyter92 commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator Author

edit: @C-Achard, I have changed the scope of this PR to small preparatory fixes. The whole pandas migration to 3.0 will entail more work than I initially estimated. This PR flags just a few examples and adds a useful tool for CI if we decide to push forward with this.

We might re-consider if the whole migration will be worth it in the first-place. Pandas 3.0 is relatively new and I doubt that users will rely on it soon, so I think we may be save with restricting to 2.x for the time being, until we eventually refactor the keypoint/dataset management in its entirety. See also tracking issue #3362

@deruyter92
deruyter92 requested a review from C-Achard June 10, 2026 09:51
@deruyter92
deruyter92 marked this pull request as ready for review June 10, 2026 09:52
Comment thread deeplabcut/core/inferenceutils.py Outdated

@C-Achard C-Achard left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very good start, just a few small comments but nothing blocking I think. Thanks for taking care of this!

Comment thread deeplabcut/gui/tracklet_toolbox.py Outdated
Comment thread deeplabcut/pose_estimation_tensorflow/core/evaluate_multianimal.py Outdated
Comment thread deeplabcut/utils/pandas_future_mode.py
Comment thread deeplabcut/utils/make_labeled_video.py
Comment thread examples/testscript_pytorch_multi_animal.py
Comment thread deeplabcut/pose_estimation_tensorflow/core/evaluate_multianimal.py Outdated
@C-Achard

Copy link
Copy Markdown
Collaborator

(Note: trying to fix current CI issues seen here in #3363 )

deruyter92 and others added 5 commits June 11, 2026 09:39
Co-authored-by: Cyril Achard <cyril.achard@epfl.ch>
Co-authored-by: Cyril Achard <cyril.achard@epfl.ch>
Co-authored-by: Cyril Achard <cyril.achard@epfl.ch>
@deruyter92

Copy link
Copy Markdown
Collaborator Author

@C-Achard Thanks for the review. Good points, I've applied all your suggestions.

@MMathisLab
MMathisLab merged commit 59a292a into main Jun 25, 2026
31 checks passed
@MMathisLab
MMathisLab deleted the jaap/migrate-pd-3 branch June 25, 2026 19:51
@C-Achard C-Achard added this to the v3.0.1 milestone Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backwards compatibility issues concerning prior to current versions dependencies Pull requests that update a dependency file enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants