Skip to content

Implement support for image/png format in terminal#15184

Merged
Carreau merged 1 commit into
ipython:mainfrom
danielzgtg:feat/terminalImagePng
Apr 20, 2026
Merged

Implement support for image/png format in terminal#15184
Carreau merged 1 commit into
ipython:mainfrom
danielzgtg:feat/terminalImagePng

Conversation

@danielzgtg

Copy link
Copy Markdown
Contributor

Closes #13287

Now PIL.Image and sympy.init_printing(use_latex="png") show images. The Kitty graphics protocol is used as suggested.

Alternative implementations:

  • Putting a mime renderer extension in ipython_config.py is inconvenient, and someone requested a native feature
  • Probing for Kitty graphics support would detect any compatible terminal, but hardcoding terminal executable names avoids startup slowdown
  • Adding tmux support via Unicode-based Kitty graphics would be very complicated

Alternatives protocols:

  • iTerm2 graphics don't work in Ghostty or st
  • Sixel requires complicated rasterization and dithering

Before

text only output

After

graphical output

@danielzgtg danielzgtg force-pushed the feat/terminalImagePng branch 2 times, most recently from a57ace1 to 0b0f4c1 Compare April 8, 2026 11:41
danielzgtg added a commit to danielzgtg/sympy that referenced this pull request Apr 8, 2026
This makes `init_session` enable graphics in Kitty just like in
QTConsole. Default to white-on-black as all Kitty-compatible terminals
default to that. Also add `printable_only` option to stop `int` from
turning into images.

Requires: ipython/ipython#15184
danielzgtg added a commit to danielzgtg/sympy that referenced this pull request Apr 8, 2026
This makes `init_session` enable graphics in Kitty just like in
QTConsole. Default to white-on-black as all Kitty-compatible terminals
default to that. Also add `printable_only` option to stop `int` from
turning into images.

Requires: ipython/ipython#15184
danielzgtg added a commit to danielzgtg/matplotlib that referenced this pull request Apr 8, 2026
This allows non-GUI terminal backends like the Kitty graphics protocol
integration in IPython to work. Previously, there was just a warning and
no output. Now, the graph is rendered to Kitty-compatible terminals as
if it were a Jupyter Notebook.

Requires: ipython/ipython#15184
Inspiration: https://github.com/Kabilan108/ipython-icat/tree/main
danielzgtg added a commit to danielzgtg/matplotlib that referenced this pull request Apr 8, 2026
This allows non-GUI terminal backends like the Kitty graphics protocol
integration in IPython to work. Previously, there was just a warning and
no output. Now, the graph is rendered to Kitty-compatible terminals as
if it were a Jupyter Notebook.

Requires: ipython/ipython#15184
Inspiration: https://github.com/Kabilan108/ipython-icat/tree/main
@danielzgtg danielzgtg force-pushed the feat/terminalImagePng branch from 0b0f4c1 to bc40447 Compare April 8, 2026 15:16
@danielzgtg

Copy link
Copy Markdown
Contributor Author

Effects on Matplotlib

No Matplotlib changes are required in this version, which uses %matplotlib inline not plt.show().

This allows non-GUI terminal backends like the Kitty graphics protocol integration in IPython to work. Previously, there was just a warning and no output. Now, the graph is rendered to Kitty-compatible terminals as if it were a Jupyter Notebook.

Inspiration: https://github.com/Kabilan108/ipython-icat/tree/main

  • Why is this change necessary? This graphics had been invisible in terminal, leaving only a NonGuiException warning. Additionally, some people are running Matplotlib in headless environments. This change makes the graphics visible for us.
  • What problem does it solve? In the terminal, closing the popup GUI window causes the graph to be lost, unlike Jupyter Notebook where it stayed in the scrollback. I also need a minimal-dependency implementation because some projects have tricky venvs with conflicting Qt and other dependencies.
  • What is the reasoning for this implementation? Previous requests were rejected for putting terminal code in "core matplotlib". Forwarding the request to IPython provides the necessary level of abstraction, simplification, and code deduplication.

Before

matplotlib descirption only no graphics in terminal

After

matplotlib plot visible in terminal

@Carreau

Carreau commented Apr 10, 2026

Copy link
Copy Markdown
Member

cursory review from phone, and comment to myself as well for later: that's cool; and thanks for doing it properly using the hooks. Maybe it should be gated behind an option that depends on wether the user set their terminal emulator instead of enabling kitty by default? i'm going to assume support for other emulator may pop up as well. i know there is an issue open to implement that; i need to check it's properly crosslinked. i also don't know which is better the octal escape code or the hex ones.

any way to detect we are in kitty ?

@danielzgtg

Copy link
Copy Markdown
Contributor Author

[...] any way to detect we are in kitty ?

I have a check for ancestor process names at https://github.com/ipython/ipython/pull/15184/changes#diff-1ebf983c373f968661eca7f4cd9dbf65708d599818fc59a33131ce14825049a3R20-R25 .

For macOS support, can someone post the output of ps aux inside iTerm2 please?

This feature isn't enabled for most users by default. Most people are either on Windows (not right platform) or in Jupyter Notebook (not isatty), where this gets disabled. Most Linux users use gnome-terminal/ptyxis, where I don't enable this either. Btw, I'm using Konsole instead of Kitty, but I tested on both and it's similar. There might be a false positive if someone starts a non-compatible terminal from a compatible terminal, but querying the terminal for its capabilities via control sequences might be too slow. In the rare event that a problem occurs, users should just put c.TerminalInteractiveShell.mime_renderers = {} in ~/.ipython/profile_default/ipython_config.py.

@Carreau

Carreau commented Apr 13, 2026

Copy link
Copy Markdown
Member
In [12]: while pp:
    ...:     pp = pp.parent()
    ...:     print(pp)
    ...:
psutil.Process(pid=30160, name='zsh', status='running', started='2026-03-25 10:14:37')
psutil.Process(pid=30159, name='login', status='sleeping', started='2026-03-25 10:14:37')
psutil.Process(pid=979, name='iTermServer-3.6.9', status='running', started='2026-03-19 10:52:02')
psutil.Process(pid=679, name='iTerm2', status='running', started='2026-03-18 00:50:59')
psutil.Process(pid=1, name='launchd', status='sleeping', started='1970-01-01 01:05:13')
psutil.Process(pid=0, name='kernel_task', status='idle', started='1970-01-01 01:05:13')
None

@Carreau

Carreau commented Apr 13, 2026

Copy link
Copy Markdown
Member

II was thinking of an ENQ request/reply (so that it work via SSH, but iterm2 does not reply to enq, and we can set do that as a subsequent PR anyway.

Closes ipython#13287

Now `PIL.Image` and `sympy.init_printing(use_latex="png")` show images.
The Kitty graphics protocol is used as suggested.

Alternative implementations:
* Putting a mime renderer extension in `ipython_config.py` is
  inconvenient, and someone requested a native feature
* Probing for Kitty graphics support would detect any compatible
  terminal, but hardcoding terminal executable names avoids startup
  slowdown
* Adding tmux support via Unicode-based Kitty graphics would be very
  complicated

Alternatives protocols:
* iTerm2 graphics don't work in Ghostty or st
* Sixel requires complicated rasterization and dithering
@danielzgtg danielzgtg force-pushed the feat/terminalImagePng branch from c06d6a4 to 1ec6bc9 Compare April 13, 2026 15:11
@danielzgtg

Copy link
Copy Markdown
Contributor Author

psutil.Process(pid=979, name='iTermServer-3.6.9', status='running', started='2026-03-19 10:52:02')
psutil.Process(pid=679, name='iTerm2', status='running', started='2026-03-18 00:50:59')

To avoid introducing a separate .startswith, I'm checking for "iTerm2". I hope I'm correct in assuming the "iTermServer-..." process never gets orphaned during normal operation.

@Carreau

Carreau commented Apr 15, 2026

Copy link
Copy Markdown
Member

Just notting something, apparently you can look at $TERM_PROGRAM that a few terminal emulator will set; I don't know which.

@danielzgtg

Copy link
Copy Markdown
Contributor Author

$TERM_PROGRAM

That is not set in Konsole and Kitty—the two terminals I'm focusing on.

@Carreau

Carreau commented Apr 20, 2026

Copy link
Copy Markdown
Member

Let's try; it's self contained enough.

@Carreau Carreau merged commit be4179d into ipython:main Apr 20, 2026
22 checks passed
@Carreau Carreau added this to the 9.13 milestone Apr 20, 2026
Carreau added a commit that referenced this pull request Apr 23, 2026
- Fill out whatsnew/version9.rst with all 6 milestone PRs: terminal
  image rendering via Kitty protocol (#15184), Python 3.11 support
  restoration (#15175), theme-aware color fix (#15156), CapturedIO
  type annotation fix (#15172), docs and contributing improvements
- Mark release.py as a full release (_version_extra = "")

https://claude.ai/code/session_016yXG8tqxaMuYxyw2bqZBEP
pull Bot pushed a commit to Stars1233/ipython that referenced this pull request Apr 23, 2026
- Fill out whatsnew/version9.rst with all 6 milestone PRs: terminal
  image rendering via Kitty protocol (ipython#15184), Python 3.11 support
  restoration (ipython#15175), theme-aware color fix (ipython#15156), CapturedIO
  type annotation fix (ipython#15172), docs and contributing improvements
- Mark release.py as a full release (_version_extra = "")

https://claude.ai/code/session_016yXG8tqxaMuYxyw2bqZBEP
Carreau added a commit that referenced this pull request May 5, 2026
Congratulations on 9.13. 🎉 

This builds on #15184 by adding an environment marker to the hard
`psutil` dependency for emscripten environments. To my knowledge, no
current approach offers processes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Revive PR : Support for image/png format in the terminal

2 participants