TST: add tests for _convolveNd_c (from convolution sub-package) C extension#19470
TST: add tests for _convolveNd_c (from convolution sub-package) C extension#19470Abu-bakkar-siddique wants to merge 8 commits intoastropy:mainfrom
Conversation
|
Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.
|
de6a221 to
c51851b
Compare
|
Should I revise the approach for the simpler contract-level cases, like kernel flipping and the bot == 0 fallback? Using the oracle there now feels a bit heavy, and explicit expected values might make those tests clearer. I still think the oracle is useful for the broader regression coverage |
neutrinoceros
left a comment
There was a problem hiding this comment.
Hi ! thank you for tacking this. It seems like a valuable set of tests. I couldn't review everything in one sitting though, but I do have a couple points I'd like you to clarify for me, because I am not very familiar with this specific function. I also want to note I stopped my review on a particular point where I was not sure the approach was sound, as it seemed more like touching on implementation details rather than operational correctness. I'd appreciate clarifications there too. Will continue my review when you reply. Thanks again !
| return result | ||
|
|
||
|
|
||
| def _trim_padding(embedded_result, kernel): |
There was a problem hiding this comment.
this function seems confusing. I'll go back to it once I read the rest
There was a problem hiding this comment.
_trim_padding was there only to cut away the outer zero-padded border from the embed=True result so that the test could compare just the inner region against the non-embedded output. I removed the helper entirely and now check the expected embedded output explicitly in the test itself :
expected = np.array(
[
[0.0, 0.0, 0.0, 0.0, 0.0],
[0.0, 12.0, 21.0, 16.0, 0.0],
[0.0, 27.0, 45.0, 33.0, 0.0],
[0.0, 24.0, 39.0, 28.0, 0.0],
[0.0, 0.0, 0.0, 0.0, 0.0],
],
dtype=np.float64,
)
| return embedded_result[tuple(slices)] | ||
|
|
||
|
|
||
| # this function acts as the oracle, a reference against which the outputs of the _convolveNd_c will be validated |
There was a problem hiding this comment.
I've never seen the term "oracle" used in this meaning, nor could I find a reference for it. Where does this come from ?
There was a problem hiding this comment.
| return np.zeros(shape, dtype=np.float64, order="C") | ||
|
|
||
|
|
||
| def _call_direct(image, kernel, *, nan_interpolate=False, embed=False, n_threads=1): |
There was a problem hiding this comment.
the name of the function seems a bit misleading to me. It's not a direct call precisely because it contains preliminary steps too.
There was a problem hiding this comment.
Replaced it with _run_extension_with_zero_padding(...) so the name reflects the setup being done before calling the extension.
| ), | ||
| pytest.param( | ||
| np.arange(1.0, 65.0).reshape(4, 4, 4), | ||
| np.ones((3, 3, 3), dtype=np.float64), |
There was a problem hiding this comment.
why is this the only example input with an explicit dtype ? I think it should be specified everywhere
There was a problem hiding this comment.
Agreed, I made dtype=np.float64 explicit throughout the test inputs for consistency.
| npt.assert_allclose(result, expected) | ||
|
|
||
|
|
||
| def test_asymmetric_kernel_is_flipped_before_multiplication(): |
There was a problem hiding this comment.
can you clarify, why is this behavior even expected ?
There was a problem hiding this comment.
The original C implementation access the kernel in reverse indexing as :
//see astropy/convolution/src/convolve.c Line 315:
for (ii = 0; ii < nkx; ++ii) {
val = f[i_minus_wkx + ii];
ker = g[nkx_minus_1 - ii];
...
}To be more clear:
ker = g[nkx_minus_1 - ii];That is the flip. Instead of g[ii], it reads the kernel from the end toward the beginning(same pattern exists in 2D and 3D dimension).
This particular test case is to check if this is being done correctly in case of asymmetric kernel ([1, 2, 3] as kernel yield different output when flipped as [3, 2, 1]) .
|
|
||
|
|
||
| def test_asymmetric_kernel_is_flipped_before_multiplication(): | ||
| """The direct extension should perform convolution, not correlation.""" |
There was a problem hiding this comment.
I do not understand how this explains the test or its name. I'd appreciate clarifications
There was a problem hiding this comment.
simply meaning that the direct extension should flip the (asymmetric) kernel, if it does not. that is not convolution (but correlation that slides the kernel without flipping)
please read my reply to the previous comment on this function for reference.
(perhaps i should have been clear about this whole idea in docstrings)
There was a problem hiding this comment.
okay, I see that this flipping is evidently intentional within the C code, but I do not understand the nuance between convolution and correlation you're drawing. I'm absolutely not a math expert, so please educate me :) (a reference on the topic would go a long way)
There was a problem hiding this comment.
Thanks for pointing that out, I was under the assumption of only 2 cases kernel flipped = convolution, and kernel not flipped = correlation, but the test should be broader than that, since it is a convolution extension, mentioning correlation is probably irrelevant here, I'll rename the function and the docstring.
There was a problem hiding this comment.
update: I made the following changes to this function:
- added descriptive docstring to convey test intent clearly
- test case now checks kernel flip for 1D, 2D, and 3D
- renamed the function to reflect the behavior being tested more accurately
please see commit a76c39f and let me know if you require any further clarifications
| def test_embed_true_matches_embed_false_in_inner_region_and_leaves_border_zero( | ||
| image, kernel | ||
| ): | ||
| """Embedded results should land in the padded coordinates and nowhere else.""" |
There was a problem hiding this comment.
I'm not sure exactly what "the padded coordinates" refers to.
| # The C loop only writes valid image coordinates, so the outer border should | ||
| # still contain the zeros from the initial result allocation. |
There was a problem hiding this comment.
this comment reads like you are exclusively writing the test against the existing behavior, instead of validating that the results are actually correct. Maybe the problem is only in the phrasing, but I would like to check with you first so we're clear what the actual approach is.
There was a problem hiding this comment.
I agree with that, reworked the tests to move away from the Python reference helper and now use explicit expected values.
|
Another note is that this is "competing" with #19455, which might be merged first because it's smaller and thus easier to review. We could very likely merge both contributions, but the second on to go in might need some additional effort to harmonize the sum. |
309b52b to
36792c6
Compare
|
Hi, thanks for taking the time to review this. Looking at it again after some distance, I think your concern is valid. I leaned too much into the implementation details of convolve.c, especially with I think the better direction is to simplify these tests and rely on explicit, hard-coded expected values. Thanks again for pointing this out , I’m going to rework the tests with that in mind. |
neutrinoceros
left a comment
There was a problem hiding this comment.
another pass. I still need to review the actual tests in more depth, but I'm already seeing the limits of my own understanding so I want to make sure I know what to look for. Thank you for your patience !
| result = _run_extension_with_zero_padding( | ||
| image, | ||
| kernel, | ||
| nan_interpolate=False, | ||
| embed=False, | ||
| ) |
There was a problem hiding this comment.
This is what I meant to suggest in my previous review: it's simpler to only pass arguments that modify the default behavior (that's what default values are for)
| result = _run_extension_with_zero_padding( | |
| image, | |
| kernel, | |
| nan_interpolate=False, | |
| embed=False, | |
| ) | |
| result = _run_extension_with_zero_padding(image, kernel) |
There was a problem hiding this comment.
(please apply similar changes to all other call sites of this helper function)
| ], | ||
| ) | ||
| def test_nan_without_interpolation_matches_expected_values(image, kernel, expected): | ||
| """Without NaN interpolation, any NaN in the window should propagate to the output.""" |
There was a problem hiding this comment.
might as well also state how it's expected to propagate; the behavior you're testing is a little bit more specific than that.
There was a problem hiding this comment.
I added a simple example in docstrings if that what you meant by how, please see commit a8c3586
If you meant something else please let me know.
| pytest.param( | ||
| np.ones((3, 3, 3), dtype=np.float64), | ||
| np.ones((3, 3, 3), dtype=np.float64), | ||
| np.full((3, 3, 3), np.nan, dtype=np.float64), |
There was a problem hiding this comment.
I did a double take here. This is misleading and relies on the test's body to modify the input.
I think it's actually preferable to hard code the actual input here.
You'll probably want to disable auto-formatting locally for a 3D literal array
There was a problem hiding this comment.
Also ruff formatter would not allow to diable autoformating just for the 3D literals, nor for the @pytest.mark.parametrize decorator, so had to wrap the whole function (after formating it correctly)
…_padding helper, where behavior change is not intended
…atches_expected_values
Description
This PR adds direct tests for the
_convolveNd_cC extension in the convolutionsubpackage as part of my GSoC 2026 interest in improving low-level test coverage.
Currently no tests target this extension directly , only the public Python API (
convolve.py) for this is tested.UPDATE:
I’ve updated the file to simplify the approach:
Testing
pytest astropy/convolution/tests/test_extension_convolve_nd_c.py -vcc @neutrinoceros @astrofrog @nstarman