[Bug]: Image color limits not correctly updated with set_clim() IFF color bar present AND new norm.vmin > old norm.vmax

Bug summary

I experienced some specific odd cases where I noticed that the colors of an image were not correctly set after updating the image data and setting new color limits with set_clim(). After debugging I was able to reproduce with a minimal example,..

Code for reproduction

import numpy as np
from matplotlib import pyplot as plt


# Create test data
np.random.seed(19771105)
data_0 = np.round(np.random.rand(10, 20), 1)
data_1 = data_0 + 10

# Compute data range
range_0 = data_0.min(), data_0.max()
range_1 = data_1.min(), data_1.max()

# Plot data as image; Auto adjusts colorbar scale to value range
image = plt.imshow(data_0)
cbar = plt.colorbar(image)  # NOTE: Problem does not occur if colorbar not created!

print(f'expected norm = {range_0}')
print(f'actual   norm = {image.norm.vmin, image.norm.vmax}')

# Change image data and update image color limits
image.set_data(data_1)
image.set_clim(range_1)

print(f'expected norm = {range_1}')
print(f'actual   norm = {image.norm.vmin, image.norm.vmax}')  # Incorrect!

plt.show()

Actual outcome

expected initial norm = (0.0, 1.0)
computed initial norm = (0.0, 1.0)
expected updated norm = (10.0, 11.0)
computed updated norm = (0.9, 11.0)

Expected outcome

expected initial norm = (0.0, 1.0)
computed initial norm = (0.0, 1.0)
expected updated norm = (10.0, 11.0)
computed updated norm = (10.0, 11.0)

Additional information

Only happens if color bar is present; Removing the line that creates it from code lets you see the correct image (minus color bar).
Data dependent; Only occurs if the lowest value of the new image data is higher than the highest value of the old image data (Hence +10 in the code example).

Operating system

Windows 10

Matplotlib Version

3.6.0

Matplotlib Backend

QtAgg

Python version

3.11.9

Jupyter version

Installation

pip

Workaround for this bug (based on example code):

#image.set_clim(range_1)
if image.norm.vmin < range_1[0]:
    image.norm.vmax = range_1[1]
    image.norm.vmin = range_1[0]
else:
    image.norm.vmin = range_1[0]
    image.norm.vmax = range_1[1]

@greglucas

I can confirm this on latest Matplotlib. Thanks for the clear minimal report. @greglucas I wonder if this has to do with your colorbar interactivity?

@erikdendekker

Yes, I think it does have to do with the signals on the norm callbacks. We trigger the signals when setting vmin originally, which then becomes greater than the original vmax (vmin > vmax temporarily). We emit a norm-has-changed signal and then the colorbar goes and processes values and then we have a non-singular expansion which swaps the vmin/vmax if they are inverted.

matplotlib/lib/matplotlib/colorbar.py

Lines 1104 to 1105 in 9e3ada3

    
           self.norm.vmin, self.norm.vmax = mtransforms.nonsingular( 
        
               self.norm.vmin, self.norm.vmax, expander=0.1)

@erikdendekker that is why you are seeing that behavior of the order of setting vmin/vmax mattering in your workaround.
#29522 (comment)

I'm not sure what exactly the proper fix is here. We are triggering callbacks, which trigger colorbar processing before we have finished updating all of our norm limits which the colorbar doesn't like. We could fix the set_clim() case because vmin/vmax come in together by suppressing the callbacks and only emitting one at the end. But, I think that a user should be able to directly call im.norm.vmin = 10, im.norm.vmax=11 and have that work too regardless of the order even though they are sequential calls and a user shouldn't need to worry about suppressing the callbacks. I wonder if there is anything we can do in colorbar to move the autoexpansion / limit updates around to help?

Blocked set_clim() callbacks to prevent inconsistent state (matplotli…

@erikdendekker

@erikdendekker in your workaround you have same statements in your if and else blocks,
and a simpler workaround would have been to use colorbar at last after setting the final range.

# Plot data as image; Auto adjusts colorbar scale to value range
image = plt.imshow(data_0)

print(f'expected norm = {range_0}')
print(f'actual   norm = {image.norm.vmin, image.norm.vmax}')

# Change image data and update image color limits
image.set_data(data_1)
image.set_clim(range_1)

cbar = plt.colorbar(image)  # use after final range has been set to avoid callbacks.

This happening within a single set_clim is clearly wrong and we should fix it, but I am not sure what should happen when setting the min and then the max one-at-a-time.

I think it comes down to a question of when should we enforce that the colormappable is in a consistent drawable state. The options I see are:

never, if vmin > vmax fail at draw time
at draw time, fix it if we have to when we go to render
at set/update-time, fix it before the next line of code can run

I not sure any of these are wrong (and we have behavior in the library of all three types). Currently setting im.norm.vmin to be greater than vax falls into the first category without a colorbar and the third with on which is the I think the real bug (as all three have tradeoffs but we should be consistent what ever we do).

This seems related to discussion we had about colorbars and auto-scaling for constant value data (however I am failing to find the issue), we should be consistent with our decision there.

This seems related to discussion we had about colorbars and auto-scaling for constant value data (however I am failing to find the issue),

That‘s #26307. The main point of discussion there is to which color value we should map, not so much when.

I think there's also the question of whether we could support norms with vmin > vmax (and e.g. treat them as reversing the colormap). Somewhat related to #12665 I guess, although that's a fairly old issue.

(edited typo, was initially incorrectly vmin < vmax)

You mean vmin > vmax? I'm a bit hesitant. Is there a use case where that's natural? I suspect that this may rather be indicative of a calculation or input error, and we should raise instead. If people want to explicitly reverse the colormap they should use the "..._r" version of the colormap.

For sure there are instances where the colorbar makes more sense if it is inverted, but keeping the values on the colorbar positive. Eg depth goes from 5000 to zero but you want the darkest blue to be 5000. Currently you'd need to reverse the colormap and reverse the colorbar, or use negative depths and change the ticks.

I don't think it's a big deal. Personally I just use negative depths in the colorbar, but if someone were more persnickety, you could imagine them being saved a song and dance if we allowed inverted norms.

Ah, now that I remember it; the other advantage of supporting vmin > vmax would be to draw inverted colorbars, e.g. the left half of this one (from a real paper, where D and eta are inversely related):

@erikdendekker in your workaround you have same statements in your if and else blocks, and a simpler workaround would have been to use colorbar at last after setting the final range.

For the simple code that reproduces this bug you are right of course,.. In my real-world application, image data is interactively updated from an external source and shown to the user. I just encountered unexpected behaviour in some specific cases, was able to pinpoint it to this issue, and condensed it to the simplified example.

Interestingly, the colorbar modifies the norm:

im = plt.imshow(a, vmin=0.8, vmax=0.2)
print(im.norm.vmin, im.norm.vmax)
plt.colorbar()
print(im.norm.vmin, im.norm.vmax)

Prints

0.8 0.2
0.18 0.8

This is definitively undesired. Norms itself currently raise if vmin > vmax (e.g. add im.norm(0.5) before addint the colorbar.

the other advantage of supporting vmin > vmax would be to draw inverted colorbars

This may be too clever. IMHO it makes reasoning on the mapping harder. Instead I suggest to invert the colorbar Axes.`

cb = plt.colorbar()
cb.ax.invert_yaxis()

We could even grow a kwarg on colorbar() if that's helpful. I think this is much clearer than exploiting vmin > vmax. The latter has the ambiguity that you don't know whether the vmin will still map to the lower end of the color range or to the higher end (and I believe there's no canoncial answer to that).

@timhoffm

Interestingly, the colorbar modifies the norm:
This is definitively undesired. Norms itself currently raise if vmin > vmax (e.g. add im.norm(0.5) before addint the colorbar.

@timhoffm, I agree. In my view, the colorbar should only "query" the norm and use that (interactive axes updating the norm for colorbars is separate and explicit), otherwise we have a divergence between images with/without colorbars. If we can push all of this logic over into the mappable drawing (and perhaps call that similar to draw_without_rendering instead here) then I think we could at least maintain consistency between figures with/without colorbars.

matplotlib/lib/matplotlib/colorbar.py

Lines 1097 to 1108 in 9e3ada3

    
           # transform from 0-1 to vmin-vmax: 
        
           if self.mappable.get_array() is not None: 
        
               self.mappable.autoscale_None() 
        
           if not self.norm.scaled(): 
        
               # If we still aren't scaled after autoscaling, use 0, 1 as default 
        
               self.norm.vmin = 0 
        
               self.norm.vmax = 1 
        
           self.norm.vmin, self.norm.vmax = mtransforms.nonsingular( 
        
               self.norm.vmin, self.norm.vmax, expander=0.1) 
        
           if (not isinstance(self.norm, colors.BoundaryNorm) and 
        
                   (self.boundaries is None)): 
        
               b = self.norm.inverse(b)

I've not looked into the details but I'm hesitant to move the non-singular expansion into the mappable. This ties again into #26307. It's a fundamental question how to handle singularity:

data can be singular
norms may autoscale to data and then have vmin = vmax. We could prohibit that, which means failing if the the user explicitly calculates limits imshow(a, vmin=a.min(), a.max()). But on autoscale imshow(a) we could expand (slight annoyance is that the amout of expansion would be an arbitrary choice).
If we don't prohibit, and support singular norms, this means (1) we have to define to which value everying is mapped (0?, 0.5?, 1). And then we need special handling in colorbars that they draw themselves in a particular way if the norm is singular (e.g. only ticks at both ends with the same value - maybe even put a special marker on the position that the singular norm maps to; i.e. if the norms maps to 0.5, mark the center of the colorbar).

re: supporting inverted norms (I commented on the other thread re: singular norms): assuming this does not introduce too many complications elsewhere (e.g. re: inverted colorbars), I think making norm.vmin = ...; norm.vmax = ... always work (regardless of the relative vmin/vmax values) without bothering with any special-casing would be a big plus consistency-wise and simplicity-wise.

Blocked set_clim() callbacks to prevent inconsistent state (matplotli…

Blocked set_clim() callbacks to prevent inconsistent state (#29522) (#…

Backport PR matplotlib#29590: Blocked set_clim() callbacks to prevent…

Merge pull request #29693 from meeseeksmachine/auto-backport-of-pr-29…

Navigation Menu

Repository navigation

[Bug]: Image color limits not correctly updated with set_clim() IFF color bar present AND new norm.vmin > old norm.vmax #29522

Bug summary

Code for reproduction

Actual outcome

Expected outcome

Additional information

Operating system

Matplotlib Version

Matplotlib Backend

Python version

Jupyter version

Installation

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	self.norm.vmin, self.norm.vmax = mtransforms.nonsingular(
	self.norm.vmin, self.norm.vmax, expander=0.1)

	# transform from 0-1 to vmin-vmax:
	if self.mappable.get_array() is not None:
	self.mappable.autoscale_None()
	if not self.norm.scaled():
	# If we still aren't scaled after autoscaling, use 0, 1 as default
	self.norm.vmin = 0
	self.norm.vmax = 1
	self.norm.vmin, self.norm.vmax = mtransforms.nonsingular(
	self.norm.vmin, self.norm.vmax, expander=0.1)
	if (not isinstance(self.norm, colors.BoundaryNorm) and
	(self.boundaries is None)):
	b = self.norm.inverse(b)

[Bug]: Image color limits not correctly updated with set_clim() IFF color bar present AND new norm.vmin > old norm.vmax #29522

Description

Bug summary

Code for reproduction

Actual outcome

Expected outcome

Additional information

Operating system

Matplotlib Version

Matplotlib Backend

Python version

Jupyter version

Installation

Activity

erikdendekker commented on Jan 25, 2025

jklymak commented on Jan 26, 2025

greglucas commented on Jan 26, 2025

prafulgulani commented on Feb 7, 2025

tacaswell commented on Feb 7, 2025

timhoffm commented on Feb 7, 2025

anntzer commented on Feb 9, 2025

timhoffm commented on Feb 9, 2025

jklymak commented on Feb 9, 2025

anntzer commented on Feb 9, 2025

erikdendekker commented on Feb 9, 2025

timhoffm commented on Feb 10, 2025

greglucas commented on Feb 10, 2025

timhoffm commented on Feb 10, 2025

anntzer commented on Feb 11, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions