Skip to content

Support all scipy.special ufuncs with quantities fixes #6390#19390

Open
luvyarana wants to merge 1 commit intoastropy:mainfrom
luvyarana:fix-6390-scipy-special
Open

Support all scipy.special ufuncs with quantities fixes #6390#19390
luvyarana wants to merge 1 commit intoastropy:mainfrom
luvyarana:fix-6390-scipy-special

Conversation

@luvyarana
Copy link
Copy Markdown

@luvyarana luvyarana commented Mar 10, 2026

Added quantity support for all remaining scipy.special ufuncs. They now properly convert dimensionless quantities and return dimensionless quantities. This includes functions with multiple outputs like airy and fresnel, and functions with multiple inputs.

A verification script was used to enumerate all 232 scipy.special ufuncs and confirm
their nin/nout signatures. This also revealed four pre existing misclassifications
(itj0y0, it2j0y0, iti0k0, it2i0k0) which are corrected in this PR.

After these corrections, all scipy.special ufuncs are now properly registered with
appropriate helpers for dimensionless inputs and outputs.

fixes #6390

@github-actions github-actions bot added the units label Mar 10, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

  • Do the proposed changes actually accomplish desired goals?
  • Do the proposed changes follow the Astropy coding guidelines?
  • Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
  • Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
  • Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
  • Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
  • Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
  • Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
  • At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

@luvyarana luvyarana marked this pull request as draft March 10, 2026 09:15
@pllim pllim added this to the v8.0.0 milestone Mar 10, 2026
@luvyarana
Copy link
Copy Markdown
Author

I investigated all scipy.special ufuncs and found that 172 were not registered in quantity_helper.

Since all of these operate on dimensionless inputs and return dimensionless outputs, I grouped them by their signature (nin/nout) and added helper functions accordingly.

This PR adds complete support along with test coverage.

If maintainers prefer a smaller incremental approach, I would be happy to split this into multiple PRs.

luvyarana pushed a commit to luvyarana/astropy that referenced this pull request Mar 11, 2026
@pllim

This comment was marked as resolved.

@pllim

This comment was marked as resolved.

@luvyarana luvyarana force-pushed the fix-6390-scipy-special branch 2 times, most recently from 46641a7 to 631782e Compare March 11, 2026 15:42
@luvyarana luvyarana marked this pull request as ready for review March 11, 2026 16:26
@luvyarana
Copy link
Copy Markdown
Author

This PR has been rebased onto the latest main, CI issues have been resolved, and commits have been squashed into a single commit. All checks are now passing and the PR should be ready for review.

The implementation systematically registers unsupported scipy.special ufuncs by grouping them according to their nin/nout signatures and mapping them to appropriate dimensionless helper functions.

@luvyarana luvyarana force-pushed the fix-6390-scipy-special branch from 459282e to 1e866c4 Compare March 11, 2026 16:48
@pllim pllim added the Extra CI Run cron CI as part of PR label Mar 11, 2026
@pllim pllim requested review from mhvk and namurphy March 11, 2026 17:07
@pllim
Copy link
Copy Markdown
Member

pllim commented Mar 11, 2026

Thanks! I will let units maintainer decide if this needs a What's New entry or not.

@luvyarana luvyarana force-pushed the fix-6390-scipy-special branch from 1e866c4 to 07ad76c Compare March 11, 2026 19:18
Copy link
Copy Markdown
Contributor

@mhvk mhvk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@luvyarana - thanks, this is mostly great. My comments inline are really small.

A bigger one, however, is that one does need to check carefully that the functions in fact take dimensionless input. I think most do but not all. I just looked through a bit, and found some that do not: agm creates a mean, so it should be like addition (and could use the addition helper), while cosm1 is like a cosine and should use that helper. Also, mathieu_cem takes one argument in degrees -- that would be one where quantities are particularly useful...

Anyway, I guess that unfortunately one has to go through all the function descriptions to check...

I do think it is worth a what's new, since this makes quantity support a lot more complete! (docs/whatsnew/8.0.rst).

for name in dimensionless_to_dimensionless_sps_ufuncs:
ufunc = getattr(sps, name, None)
SCIPY_HELPERS[ufunc] = helper_dimensionless_to_dimensionless
if ufunc is not None:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you do this test? Shouldn't the ufunc always exist? (Also below)

I realize that the original code here is a bit weird too in using getattr(sps, name, None) instead of getattr(sps, name) as done below.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The guard is mainly there to avoid import time failures with older
SciPy versions where some ufuncs may not be present.

I kept the getattr(..., None) pattern for consistency with existing code and to
ensure compatibility, but happy to simplify this if we want to rely on a minimum
SciPy version where all listed ufuncs are guaranteed to exist.

@pllim
Copy link
Copy Markdown
Member

pllim commented Mar 16, 2026

Looks like someone else proposed a completely different approach at #19442 (doc update only). 🤷‍♀️

@luvyarana
Copy link
Copy Markdown
Author

@mhvk Thanks, this is very helpful.

You're right that not all scipy.special functions are strictly dimensionless → dimensionless.
I'll go through the function documentation and refine the classification, handling
cases like agm, cosm1, and mathieu_cem appropriately.

I'll also add a What's New entry as suggested.

@luvyarana luvyarana force-pushed the fix-6390-scipy-special branch 3 times, most recently from dc375e2 to 27a76e8 Compare March 17, 2026 09:14
@mhvk
Copy link
Copy Markdown
Contributor

mhvk commented Mar 17, 2026

I definitely prefer to actually support the scipy ufuncs! And this PR is close, since most do require dimensionless quantities -- the only unfortunate thing is that one has to go through each of them to verify that. Thanks, @luvyarana, for taking this on!

@luvyarana
Copy link
Copy Markdown
Author

luvyarana commented Mar 17, 2026

@mhvk
Based on your feedback, I’ve made the following updates:

• Revisited the ufunc classification to better reflect their behavior.
cosm1, ellipeinc, and mathieu_* now correctly handle angle Quantity inputs,
while agm preserves units like an arithmetic operation.

• Standardized all helper registration loops to use
getattr(sps, name, None) with guards, ensuring compatibility with older SciPy versions.

• Cleaned up the code by removing unclear comments and fixing minor PEP8 issues.

• Updated behavior for key cases:

  1. cosm1 accepts angle inputs and returns dimensionless output
  2. agm preserves units and enforces compatible inputs
  3. mathieu_* accepts angle inputs and converts to degrees as required by SciPy

• Added tests for these cases — all tests and pre-commit checks pass.

(One note from verification: for mathieu_modcem/modsem, although scipy docs mention "degrees",
the functions behave as non periodic coordinate based functions. Based on numerical checks,
treating them as dimensionless is appropriate.

Similarly, for lpmv, "degree" refers to the polynomial order rather than an angle unit,
so dimensionless handling is correct.)

I’ve also added a What’s New entry for this change.

@luvyarana luvyarana force-pushed the fix-6390-scipy-special branch from f01255c to 6523033 Compare March 17, 2026 10:51
@luvyarana luvyarana requested a review from mhvk March 17, 2026 11:16
@mhvk
Copy link
Copy Markdown
Contributor

mhvk commented Mar 17, 2026

I'll look at the actual code later, but two quick replies to your longer comment now:

Standardized all helper registration loops to use
getattr(sps, name, None) with guards, ensuring compatibility with older SciPy versions.

I'd like to only use this pattern if there is an actual need. I think new functions get relatively rarely added to scipy, so checking whether a function exists should generally not be needed. It is also better to have an explicit scipy version check in that case, so we can clean up things when they are no longer needed.

(One note from verification: for mathieu_modcem/modsem, although scipy docs mention "degrees",
the functions behave as non periodic coordinate based functions. Based on numerical checks,
treating them as dimensionless is appropriate

That seems wrong. If a function expects an argument in degrees, then the point of Quantity is that if one passes in, say, arcminutes, they get properly converted.

@luvyarana luvyarana force-pushed the fix-6390-scipy-special branch from 6523033 to 0199893 Compare March 17, 2026 12:14
@luvyarana
Copy link
Copy Markdown
Author

I think new functions get relatively rarely added to scipy, so checking whether a function exists should generally not be needed. It is also better to have an explicit scipy version check in that case, so we can clean up things when they are no longer needed.

I’ve removed the getattr(..., None) guards as suggested. Just to confirm,
are we comfortable relying on the minimum supported SciPy version to
guarantee that all these ufuncs are available?

@mhvk
Copy link
Copy Markdown
Contributor

mhvk commented Mar 17, 2026

Yes, minimum scipy version is all we guarantee astropy works for; if one wants an older one, use an older astropy too...

Note that there are test failures with "oldest supported versions", so apparently there are new additions to the special functions. If there are a lot of those, I perhaps should reconsider my ideas....

@luvyarana
Copy link
Copy Markdown
Author

Ah, I see thanks, that clarifies it.

It looks like several of the ufuncs included here are not present in the oldest supported SciPy version, which explains the failures.

Given that, I think restoring the getattr(sps, name, None) + guard pattern makes sense here to maintain compatibility across supported versions, especially since the ufunc set does vary.

I can reintroduce the guard consistently across all loops does that sound like the preferred approach?

@luvyarana luvyarana force-pushed the fix-6390-scipy-special branch from a8fc8c3 to db650c8 Compare March 17, 2026 16:01
@luvyarana
Copy link
Copy Markdown
Author

The issue was that some ufuncs in the list (e.g., ellip*, wright_bessel, log_wright_bessel) are not present in the minimum supported SciPy version (1.13). Using bare getattr(sps, name) caused an AttributeError at import time, which prevented all scipy ufuncs from being registered.

I’ve restored the safe pattern:
getattr(sps, name, None) with an if ufunc is not None guard across all loops (and kept the hasattr check for round), so missing functions are skipped without breaking registration.

All tests and pre-commit checks are now passing.

Classify all scipy.special ufuncs by their physical semantics:

- Dimensionless: error functions, gamma functions, Bessel functions,
  statistical distributions, hypergeometric functions, etc.
- Angle-accepting: cosm1 (radians), ellipeinc/ellipkinc (radians),
  mathieu_cem/sem and mathieu_modcem1/2/modsem1/2 (degrees),
  cosdg/sindg/tandg/cotdg (degrees).
- Arithmetic-like: agm (arithmetic-geometric mean, preserves units).
- Special cases: cbrt (unit transform), radian (angle conversion),
  round (unit-preserving).

Use getattr(sps, name, None) guards throughout so that ufuncs added
after the minimum supported SciPy version (1.13) are silently skipped
rather than raising AttributeError at import time.

Also fix pre-existing misclassification of itj0y0, it2j0y0, iti0k0,
it2i0k0 as 1-output (they have 2 outputs).

Fixes astropy#6390
@luvyarana luvyarana force-pushed the fix-6390-scipy-special branch from cfa8e1c to 1a0761b Compare March 23, 2026 09:59
@luvyarana
Copy link
Copy Markdown
Author

Hi @mhvk just a quick ping on this.

I’ve addressed the earlier feedback, including restoring the guarded getattr pattern for compatibility with older SciPy versions. All tests and checks are now passing.

I do see a failure in the mpldev image comparison job (test_noncelestial_angular, RMS ~8.6). From what I can tell, this test is in wcsaxes and doesn’t use scipy.special, so it seems unrelated to the changes here possibly due to matplotlib dev rendering differences. But since the difference is relatively large, I wanted to double check whether this should be treated as an mpldev baseline issue or something I should investigate further.

Would appreciate another look when you get a chance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Extra CI Run cron CI as part of PR units

Projects

None yet

Development

Successfully merging this pull request may close these issues.

multiple functions from scipy.special do not work with quantities

3 participants