Skip to content

Pins numpy<2, fix svd for scipy>=1.11.0#2827

Merged
MMathisLab merged 5 commits intomainfrom
niels/pin_numpy
Jan 7, 2025
Merged

Pins numpy<2, fix svd for scipy>=1.11.0#2827
MMathisLab merged 5 commits intomainfrom
niels/pin_numpy

Conversation

@n-poulsen
Copy link
Contributor

@n-poulsen n-poulsen commented Jan 7, 2025 •
edited
Loading

This pull request pins the numpy version required for DeepLabCut to numpy<2.0.0 and fixes SVD computation in Tracklet.estimate_rank for scipy>=1.11.0.

scipy SVD computation

With scipy<1.11.0, computation of the SVD of an all-zero matrix would be successful, returning an all-zero array for the singular values. With scipy>=1.11.0, this fails with a ValueError. Hence, we first check if the matrice is the zero matrix before computing the SVD. If it is, we return a zero-vector to match the behavior of scipy<1.11.0.

This can be verified with the following script:

import numpy as np
from scipy.linalg.interpolative import svd

mat = np.ones((100, 100))
u, s, v = svd(mat, 10)
print(s.shape)
print(s)

mat = np.zeros((100, 100))
u, s, v = svd(mat, 10)
print(s.shape)
print(s)

With scipy==1.10.1 this succeeds with the output:

# Installed packages
# 
# Package    Version
# ---------- -------
# numpy      1.26.4
# pip        24.2
# scipy      1.10.1
# setuptools 75.1.0
# wheel      0.44.0

(10,)
[1.00000000e+002 2.00894856e-014 9.74976675e-030 1.87711978e-044
 3.36274201e-059 7.64750134e-074 6.78987388e-089 1.85753577e-103
 5.94236148e-118 5.94807826e-133]
(10,)
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]

While with scipy==1.15.0 this fails with the output:

# Installed packages
# 
# Package    Version
# ---------- -------
# numpy      1.26.4
# pip        24.2
# scipy      1.15.0
# setuptools 75.1.0
# wheel      0.44.0

(10,)
[1.00000000e+002 1.97841743e-014 2.33093502e-029 3.08938377e-044
 4.24604188e-059 6.43087871e-074 1.13626229e-088 8.36470319e-104
 1.62555162e-118 3.13271339e-133]
Traceback (most recent call last):
  File "sktest.py", line 11, in <module>
    u, s, v = svd(mat, 10)
  File "/miniconda/envs/sk15/lib/python3.10/site-packages/scipy/linalg/interpolative.py", line 905, in svd
    U, S, V = _backend.iddr_asvd(A, k, rng=rng)
  File "_decomp_interpolative.pyx", line 933, in scipy.linalg._decomp_interpolative.iddr_asvd
  File "/miniconda/envs/sk15/lib/python3.10/site-packages/scipy/linalg/_decomp_svd.py", line 106, in svd
    a1 = _asarray_validated(a, check_finite=check_finite)
  File "/miniconda/envs/sk15/lib/python3.10/site-packages/scipy/_lib/_util.py", line 537, in _asarray_validated
    a = toarray(a)
  File "/miniconda/envs/sk15/lib/python3.10/site-packages/numpy/lib/function_base.py", line 630, in asarray_chkfinite
    raise ValueError(
ValueError: array must not contain infs or NaNs

@n-poulsen n-poulsen added the dependencies Pull requests that update a dependency file label Jan 7, 2025
@n-poulsen n-poulsen requested a review from MMathisLab January 7, 2025 14:00
@n-poulsen n-poulsen changed the title Pins numpy<2 Pins numpy<2, fix stitching for scipy>=1.11.0 Jan 7, 2025
@n-poulsen n-poulsen changed the title Pins numpy<2, fix stitching for scipy>=1.11.0 Pins numpy<2, fix svd for scipy>=1.11.0 Jan 7, 2025
@n-poulsen n-poulsen requested a review from maximpavliv January 7, 2025 15:22
@MMathisLab MMathisLab merged commit d4da23a into main Jan 7, 2025
5 checks passed
@n-poulsen n-poulsen deleted the niels/pin_numpy branch January 7, 2025 15:28
@n-poulsen n-poulsen mentioned this pull request Jan 7, 2025
Closed
2 tasks
n-poulsen added a commit that referenced this pull request Jan 8, 2025
@n-poulsen
Pins numpy<2, fix svd for scipy>=1.11.0 (#2827)
20b6e49 
* pin numpy to < 2.0

* update numpy pin in requirements.txt

* fix SVD in stitch with scipy>=1.11.0

* speedup check if matrix contains only zeros

* improve readability
@maximpavliv maximpavliv mentioned this pull request Jan 10, 2025
Closed
@hmaarrfk
Copy link
Contributor

hmaarrfk commented Feb 17, 2025

As a user and downstream packager, I would just like to ask why numpy is being pinned to less than 2?

In my experience it has been quite "smooth" all things considered to transition to numpy2 for python packages. Tensorflow also just released a version compatible with numpy2 these last few months (tensorflow 2.18)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@MMathisLab MMathisLab MMathisLab approved these changes

@maximpavliv maximpavliv Awaiting requested review from maximpavliv

Assignees

No one assigned

Labels

dependencies Pull requests that update a dependency file

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@n-poulsen @hmaarrfk @MMathisLab