Repository navigation
Coerce MxNx1 images into MxN images for imshow - #15090
Conversation
|
I could easily add a unit test to |
|
This is projecting (N, M, 1) input down to (N, M) which means in goes through color mapping code is by default produces an image which is not grayscale. The other option would be to project it up to a (N, M, 3) array where each plane is identical which would run the input through the RGB(A) handling code path which would produce a gray-scale output. My inclination would be towards the first case, but from your language motivating this change I expected it to be doing the second. |
|
Thank you for working on this! Either way, we will definitely need a test and a whats_new or API change entry. |
|
As mentioned in the issue I think it’s confusing for mxnx1 array to behave differently from an mxn array. If anything, I think we should just reshape the mxnx1 array to drop the redundant dimension. |
adding @tacaswell input Co-Authored-By: Thomas A Caswell <tcaswell@gmail.com>
|
@tacaswell @jklymak for the usual use cases with tensorflow and pytorch it would make more sense to treat it the same way as the N, M (rather than N, M, 3) since it is often scaled differently. For me the main point would be that all model outputs could be handled the same way rather than having add |
adding a test to make sure it works correctly with a 10x10x1 image
|
For your current code, |
|
@tacaswell that is potentially a perfectly acceptable solution.Should I close the PR? |
|
We reshape arrays in other contexts. I think coercing MxNx1 to MxN is reasonable in this case. |
removing whitespace
adding whats_new document
updating based on feedback from @ImportanceOfBeingErnest
|
I think ultimately one would like to test each of the following cases
It doesn't look like those are all tested currently. An image comparisson test may make sense for cases 1, 4 and 5. (All of those could be tested within the same figure) For case 2. one may rather compare the output to case 1., using the |
|
@ImportanceOfBeingErnest I hadn't known about the |
|
That would be the ultimate goal. Up to you if you want to do that work and include it in this PR or not. |
changing to check_figures_equal, adding 10x10x2 and 10x10x5 failure cases
|
The x3 and x4 cases seem to be already covered in other tests, so that should matplotlib/lib/matplotlib/tests/test_image.py Lines 49 to 59 in 22a3392 matplotlib/lib/matplotlib/tests/test_image.py Lines 276 to 298 in 22a3392 |
|
@ImportanceOfBeingErnest everything seems fine for me (but I am very inclined to say that), anything else I should add? |
|
Appart from the flake8 error this looks fine to me. |
removing whitespace
|
@ImportanceOfBeingErnest I can change it, it appears to be a completely different mechanism in |
|
I guess that can also be another PR; depending on what is easiest. Approving this as it stands. |
| "float".format(self._A.dtype)) | ||
|
|
||
| if (self._A.ndim == 3 and self._A.shape[-1] == 1): | ||
| # If just one dimension assume grayscale |
There was a problem hiding this comment.
This comment isn't correct anymore - its not assuming greyscale, its just doing a mapping into the colormap?
There was a problem hiding this comment.
as that slipped past me, thanks!
updated comment
Zac-HD
left a comment
There was a problem hiding this comment.
I'm -0 on this change - I'd prefer to add a more helpful error message with the .squeeze(axis=-1) trick, because IMO building in fixes like this tends to make it harder to develop a mental model of what the arrays should be doing or what counts as valid input. This would be far from the only case though, so it's not a strong objection!
@kmader - add "Closes #15089" to your top-level description, and it will auto-close when this is merged 😄
|
|
||
| if (self._A.ndim == 3 and self._A.shape[-1] == 1): | ||
| # If just one dimension assume scalar and apply colormap | ||
| self._A = self._A[:, :, 0] |
There was a problem hiding this comment.
self._A = self._A.squeeze(axis=-1)|
Pre-emptively labeling as release critical to make sure it does not fall through the cracks. The discussion at #7221 suggests that at least some users would expect us to go to RGB gray scale rather than color mapping. |
|
@tacaswell are you suggesting we hold off on merging it? I'm not sure why MxNx1 would be considered greyscale, but MxN would be considered color mappable. Is this really a particular convention in machine learning? I'm not clear on why that would be the case; I can imagine having a 3-D array MxNxNsamps, but I'd have expected that to be accessed as |
|
+/-0 on accepting MxNx1. On the one hand, we should have clear APIs. On the other hand, it may be convenient to accept MxNx1 specifically, if people have this kind of data a lot. However, if MxNx1 is accepted, it should be handled the same way as MxN. I don't think it's safe to assume that MxNx1 is a grayscale image ("In the face of ambiguity, refuse the temptation to guess."). Let's rather go for consistency than a subtle different behavior, which may be handy for some, but unexpected for others. |
|
My main reason for adding it was my experience from teaching students newer to python where particularly with segmentation problems students were confused at why MxN and MxNx3 images worked but MxNx1 didn't. I can, however, appreciate that there are few easy decisions. |
|
Right but how did they get an mxnx1 matrix in the first place? |
|
PyTorch, Keras, and Tensorflow, and even simpler tools like https://github.com/Calysto/conx all output binary segmentations (the most common case) as MxNx1 |
|
Is there a good reason why they do this? |
|
The main reason (AFAIK) they do this is to make convolutions in convolutional neural networks easier. So all the convolutions have MxNxQ to MxNxR format and binary is just when R=1 |
|
Cool, so from my point of view there is no special difference between MxNx1 and MxN, so I still think this PR that treats them as the same is a reasonable convenience. But I don't feel strongly about it if there are strong objections. |
|
Looks good to me as well. I also asked @jni as someone who works on images if he thought there was a case where we wouldn't want this, and he couldn't find any. |
PR Summary
Causes
matplotlib.pyplot.imshowto work with depth=1 images (common in deep learning and segmentation) as if they were grayscale rather than throwing an error message (so it should be backward compatible, unless someone expected it to throw an error)PR Checklist
Specific Tests