Skip to content

Adds show_in_notebook functions for Interactive tables using itables#7899

Merged
nabobalis merged 51 commits into
sunpy:mainfrom
Prtm2110:searchable_tables
Oct 30, 2025
Merged

Adds show_in_notebook functions for Interactive tables using itables#7899
nabobalis merged 51 commits into
sunpy:mainfrom
Prtm2110:searchable_tables

Conversation

@Prtm2110
Copy link
Copy Markdown
Contributor

@Prtm2110 Prtm2110 commented Dec 5, 2024

PR Description

Fixes #7639

@Prtm2110 Prtm2110 requested a review from a team as a code owner December 5, 2024 07:49
Comment thread pyproject.toml Outdated
Comment thread sunpy/net/fido_factory.py Outdated
Comment thread sunpy/net/fido_factory.py Outdated
Comment thread sunpy/net/tests/test_fido.py Outdated
@Prtm2110
Copy link
Copy Markdown
Contributor Author

Prtm2110 commented Dec 8, 2024

itables can be a great alternative here, as it eliminates the need for additional dependencies. Here's how you can use it:

from itables import show
show(df)

If you prefer to avoid the IPython dependency, you can consider the following approach:
Example Screenshot

Additionally, do you have any suggestions for writing tests for this integration?

@nabobalis
Copy link
Copy Markdown
Member

nabobalis commented Dec 8, 2024

itables can be a great alternative here, as it eliminates the need for additional dependencies. Here's how you can use it:

I assume astropy picked the other library for a reason?

What is the difference between them?

Also is this an AI answer?

Additionally, do you have any suggestions for writing tests for this integration?

Honestly, I do not. How did astropy do it?

Since astropy are dealing with these types of dependencies, I think investigating how they do that and copying their solution is the simplest path forward.

@Prtm2110
Copy link
Copy Markdown
Contributor Author

Prtm2110 commented Dec 8, 2024

I assume astropy picked the other library for a reason?

I believe maybe they didn't pick itables because "itables appears to change how dataframes are rendered Notebook-wide which I felt isn't something we're going for." said by exitflynn, although this is not a problem now because itable have show(df) method which only converts particular dataframes rather than all.

What is the difference between them?

If we use itables we can directly use their show method which can display them in jupyter notbook, but in astropy they have used ipydatagrid to single tables and returned them directly, which we cannot do in sunpy as there can be multiple tables in a fido result, also this is the reason why we cannot use astropy's show_in_notebook method.

Edit: itables looks much better and feels faster to use.

Also is this an AI answer?

Neither the code nor the comments are ai (although I use grammarly), I guess it is a problem? I will stop that now.

Additionally, do you have any suggestions for writing tests for this integration?

Honestly, I do not. How did astropy do it?

Since astropy are dealing with these types of dependencies, I think investigating how they do that and copying their solution is the simplest path forward.

Yes, they just checked isInstance(t.show_in_table(), Datagrid) but again this wouldn't work here, because we are not returning Datagrid as they are.

@nabobalis
Copy link
Copy Markdown
Member

I assume astropy picked the other library for a reason?

I believe maybe they didn't pick itables because "itables appears to change how dataframes are rendered Notebook-wide which I felt isn't something we're going for." said by exitflynn, although this is not a problem now because itable have show(df) method which only converts particular dataframes rather than all.

How does it change them compared to the other library?

What is the difference between them?

If we use itables we can directly use their show method which can display them in jupyter notbook, but in astropy they have used ipydatagrid to single tables and returned them directly, which we cannot do in sunpy as there can be multiple tables in a fido result, also this is the reason why we cannot use astropy's show_in_notebook method.

So how does both libraries handle having multiple tables and rendering them in a notebook?

Also is this an AI answer?

Neither the code nor the comments are ai (although I use grammarly), I guess it is a problem? I will stop that now.

No that's fine. It's a common tool and its different from Gen AI.

Additionally, do you have any suggestions for writing tests for this integration?

Honestly, I do not. How did astropy do it?
Since astropy are dealing with these types of dependencies, I think investigating how they do that and copying their solution is the simplest path forward.

Yes, they just checked isInstance(t.show_in_table(), Datagrid) but again this wouldn't work here, because we are not returning Datagrid as they are.

Hmm, I expect that properly testing this kind of output is painful and why it was not done. Can you check how the quicklook method is tested inside sunpy?

@nabobalis nabobalis marked this pull request as draft December 8, 2024 21:36
@Prtm2110
Copy link
Copy Markdown
Contributor Author

Prtm2110 commented Dec 9, 2024

I assume astropy picked the other library for a reason?

I believe maybe they didn't pick itables because "itables appears to change how dataframes are rendered Notebook-wide which I felt isn't something we're going for." said by exitflynn, although this is not a problem now because itable have show(df) method which only converts particular dataframes rather than all.

How does it change them compared to the other library?

itable has this feature as well

from itables import init_notebook_mode

init_notebook_mode(all_interactive=True)

This converts all dataframes into those interactive tables, this is what exitflynn meant although i guess he missed or they added show(df) new method later, which makes changes to particular dataframes only

What is the difference between them?

If we use itables we can directly use their show method which can display them in jupyter notbook, but in astropy they have used ipydatagrid to single tables and returned them directly, which we cannot do in sunpy as there can be multiple tables in a fido result, also this is the reason why we cannot use astropy's show_in_notebook method.

So how does both libraries handle having multiple tables and rendering them in a notebook?

Naturally they cannot render multiple tables, but itable renders the tables in distinct boxes with search button (image in the comment above) which looks good.

Additionally, do you have any suggestions for writing tests for this integration?

Honestly, I do not. How did astropy do it?
Since astropy are dealing with these types of dependencies, I think investigating how they do that and copying their solution is the simplest path forward.

Yes, they just checked isInstance(t.show_in_table(), Datagrid) but again this wouldn't work here, because we are not returning Datagrid as they are.

Hmm, I expect that properly testing this kind of output is painful and why it was not done. Can you check how the quicklook method is tested inside sunpy?

Yes I had a look, I will try using mocking.

@nabobalis
Copy link
Copy Markdown
Member

Naturally they cannot render multiple tables, but itable renders the tables in distinct boxes with search button (image in the comment above) which looks good.

So in your opinion, which library should we use?

Yes I had a look, I will try using mocking.

Thanks

@Prtm2110
Copy link
Copy Markdown
Contributor Author

Prtm2110 commented Dec 11, 2024

Naturally they cannot render multiple tables, but itable renders the tables in distinct boxes with search button (image in the comment above) which looks good.

So in your opinion, which library should we use?

itables, for some reasons I am not able to run the test locally I am pushing to test it.
Let me know if we are sticking with ipydatagrid, I will make changes accordingly and fix pre-commit and changelog

@Prtm2110
Copy link
Copy Markdown
Contributor Author

Also can you please clarify what you mean by attrs table, and how they are generated, a code example maybe?

@nabobalis
Copy link
Copy Markdown
Member

Also can you please clarify what you mean by attrs table, and how they are generated, a code example maybe?

https://docs.sunpy.org/en/stable/tutorial/acquiring_data/index.html#searching-for-data

@Prtm2110
Copy link
Copy Markdown
Contributor Author

Also can you please clarify what you mean by attrs table, and how they are generated, a code example maybe?

https://docs.sunpy.org/en/stable/tutorial/acquiring_data/index.html#searching-for-data

image
Is the position and the name of search function acceptable? I have created a _create_table function to reduce redundancy of code

@nabobalis
Copy link
Copy Markdown
Member

Looks fine with me.

@Prtm2110
Copy link
Copy Markdown
Contributor Author

Prtm2110 commented Apr 9, 2025

  1. Why are we using itables over ipydatagrid? Both seem relatively popular but the former is maintained by a single person and the latter is maintained by the Jupyter project.

Hi, ipydatagrid felt much slower and more laggy when I used it, which is why we decided to go with tables. Ref comment

  1. How are we handling the itables/ipydatagrid dependency? Is an additional optional dependency category for notebook-related things being added?

I'm not quite sure could you please let me know where the dependency should be placed

@Prtm2110 Prtm2110 requested a review from Cadair April 25, 2025 08:41
@Cadair Cadair requested a review from nabobalis October 29, 2025 12:30
@Cadair
Copy link
Copy Markdown
Member

Cadair commented Oct 29, 2025

2. How are we handling the itables/ipydatagrid dependency? Is an additional optional dependency category for notebook-related things being added?

I've now added an optional jupyter extra which also has ipywidgets in it for the parfive progress bars.

@Cadair Cadair requested a review from wtbarnes October 29, 2025 12:32
Comment thread changelog/7899.feature.rst Outdated
Comment thread sunpy/net/tests/test_fido.py Outdated
assert isinstance(results[0], QueryResponseTable)


@pytest.mark.remote_data
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know we have random requirements on remote data but do we really need to use a real fido return for this

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I agree, there's no need for this to be an online PR.

Copy link
Copy Markdown
Member

@Cadair Cadair left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make the test not online.

Comment thread docs/tutorial/installation.rst Outdated
@nabobalis nabobalis requested review from Cadair and removed request for nabobalis October 30, 2025 17:43
@nabobalis nabobalis removed the Needs Review Needs review(s) before merge. label Oct 30, 2025
@nabobalis
Copy link
Copy Markdown
Member

Changed test to be offline.

@nabobalis nabobalis merged commit 290d439 into sunpy:main Oct 30, 2025
27 of 30 checks passed
@nabobalis
Copy link
Copy Markdown
Member

Thank you again @Prtm2110

@Prtm2110 Prtm2110 deleted the searchable_tables branch October 30, 2025 19:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

net Affects the net submodule No Backport A PR that isn't to be backported to any release branch. (To be used as a flag to other maintainers) Whats New? Needs a section added to the current Whats New? page.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make the attrs table searchable

5 participants