You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Bug]: MacOSX backend windows have no accessibility data #24608
A very common way of interacting with windows on MacOS is through Apple's Accessibility Framework, which is used for all manner of screen manipulations, voice readers, etc. For example, sight-impaired people can use voice commands to select buttons or have control names read to them, users can use accessibility controls to move and arrange windows, etc.
I have discovered that the plotting windows produced in the "Python" app of the MacOSX backend are created without any accessibility information whatsoever. I used the "Accessibility Inspector" to check this. I've never encountered this before; in fact I didn't think it was possible. Is it a known issue?
@jdtsmith, I've never used the Accessibility Inspector before, is there something I need to do to get it to capture the window properly? It doesn't register anything for me on any of the backends. I have been running with MPLBACKEND=qt5agg python test.py (running with ipython --matplotlib=qt5 and inputting interactively didn't work for me either.
We are including the ApplicationServices header, but maybe there is some more explicit work we need to do in the macosx backend. PRs very welcome on this!
You hit option-space and that activates the cursor to grab any elements (which is highlighted in green). Clicking on an element then "freezes" it so you can inspect the information. You'll notice that the macosx backend plot window is invisible to Accessibility Inspector.
Unfortunately I've never implemented such a thing in Objective C; just a "user" of accessibility data via a tool called Hammerspoon.
You'll notice that the macosx backend plot window is invisible to Accessibility Inspector.
That is what all of the Matplotlib backends look like to me :( The highlighted green selector gets all of the surrounding windows but doesn't ever see the Matplotlib ones as you mention.
Unfortunately I've never implemented such a thing in Objective C; just a "user" of accessibility data via a tool called Hammerspoon.
If you are interested in improving Matplotlib's accessibility in other ways or chiming in with advice, we have a few current issues surrounding alt-text and ARIA where your input would be welcome! #21328 #24309
You'll notice that the macosx backend plot window is invisible to Accessibility Inspector.
That is what all of the Matplotlib backends look like to me :( The highlighted green selector gets all of the surrounding windows but doesn't ever see the Matplotlib ones as you mention.
Maybe that's good news? I.e. it means PyQT recently added it? Here are my versions:
I think the root of the problem is which compiled python we are using to run our programs. To get accessibility features we need a Framework build of Python.
If I create a new virtual environment from a brew-installed Python 3.11 and use that, I get the accessibility information on all of the buttons and windows.
However, if I use a conda environment's python (which is non-framework), then the window is invisible to the accessibility panel as you described. If you are in conda, you can choose a framework build of Python with "w" pythonw test_script.py which then also shows the accessibility information.
Unfortunately, I don't think there is anything we can do on the Matplotlib side for this.
Note that QT backends offer accessibility with non-framework python versions. It's not clear how they manage that yet the MacOSX backend does not.
I can't reproduce that with the same pyqt versions as you posted above. I need a framework python version to get QT and Tk windows to show up in the accessibility framework as well. How are you installing python and matplotlib for the different situations?
I take that back. I am indeed using a framework build (brew-installed), yet the MacOS backend does not provide accessibility information, unlike QT. This is Python 3.11.3, MPL 3.7.1.
Odd, and you are running the same python with each of the backends? Here is a quick screencap with a fresh install of matplotlib==3.7.1 bringing up a simple figure window with the macosx backend and then showing the accessibility options that appear on the right when clicking the pan button.
Some other thoughts for differences: M1/arm Mac (I'm on intel), XCode/clang versions.
Activity
greglucas commentedon Dec 5, 2022
I don't think this is exclusive to the macosx backend. I can't pick up anything with
tkaggorqt5aggbackends either.jdtsmith commentedon Dec 5, 2022
Thanks. With
--matplotlib=qt5in iPython, I do find accessibility info on the plot window:It's much slower than macosx backend though, so it would be great to add accessibility data to that.
greglucas commentedon Dec 6, 2022
@jdtsmith, I've never used the Accessibility Inspector before, is there something I need to do to get it to capture the window properly? It doesn't register anything for me on any of the backends. I have been running with
MPLBACKEND=qt5agg python test.py(running withipython --matplotlib=qt5and inputting interactively didn't work for me either.We are including the ApplicationServices header, but maybe there is some more explicit work we need to do in the macosx backend. PRs very welcome on this!
matplotlib/src/_macosx.m
Line 3 in b080109
jdtsmith commentedon Dec 6, 2022
You hit option-space and that activates the cursor to grab any elements (which is highlighted in green). Clicking on an element then "freezes" it so you can inspect the information. You'll notice that the macosx backend plot window is invisible to Accessibility Inspector.
Unfortunately I've never implemented such a thing in Objective C; just a "user" of accessibility data via a tool called Hammerspoon.
greglucas commentedon Dec 6, 2022
That is what all of the Matplotlib backends look like to me :( The highlighted green selector gets all of the surrounding windows but doesn't ever see the Matplotlib ones as you mention.
If you are interested in improving Matplotlib's accessibility in other ways or chiming in with advice, we have a few current issues surrounding alt-text and ARIA where your input would be welcome!
#21328
#24309
jdtsmith commentedon Dec 6, 2022
Maybe that's good news? I.e. it means PyQT recently added it? Here are my versions:
greglucas commentedon Jun 4, 2023
I think the root of the problem is which compiled
pythonwe are using to run our programs. To get accessibility features we need a Framework build of Python.If I create a new virtual environment from a brew-installed Python 3.11 and use that, I get the accessibility information on all of the buttons and windows.
python3.11 -m venv .env source .env/bin/activate pip3 install matplotlib==3.7.1 python3 test_script.pyHowever, if I use a conda environment's python (which is non-framework), then the window is invisible to the accessibility panel as you described. If you are in conda, you can choose a framework build of Python with "w"
pythonw test_script.pywhich then also shows the accessibility information.Unfortunately, I don't think there is anything we can do on the Matplotlib side for this.
jdtsmith commentedon Jun 5, 2023
Note that QT backends offer accessibility with non-framework python versions. It's not clear how they manage that yet the MacOSX backend does not.
greglucas commentedon Jun 5, 2023
I can't reproduce that with the same pyqt versions as you posted above. I need a framework python version to get QT and Tk windows to show up in the accessibility framework as well. How are you installing python and matplotlib for the different situations?
jdtsmith commentedon Jun 5, 2023
I take that back. I am indeed using a framework build (brew-installed), yet the MacOS backend does not provide accessibility information, unlike QT. This is Python 3.11.3, MPL 3.7.1.
greglucas commentedon Jun 5, 2023
Odd, and you are running the same python with each of the backends? Here is a quick screencap with a fresh install of matplotlib==3.7.1 bringing up a simple figure window with the macosx backend and then showing the accessibility options that appear on the right when clicking the pan button.

Some other thoughts for differences: M1/arm Mac (I'm on intel), XCode/clang versions.