refactor: Enable SELECT * optimization in the sqlglot compiler
#2430
Conversation
product-auto-label
bot
added
size: m
chelsea-lin
force-pushed
the
main_chelsealin_selectstart
branch
from
e085050 to
f5a2276
Compare
| nodes.ScanItem( | ||
| identifiers.ColumnId(scan_item.source_id), scan_item.source_id | ||
| ) | ||
| for scan_item in node.scan_list.items |
There was a problem hiding this comment.
I think what we really want is to order by the underlying physical schema?
There was a problem hiding this comment.
When generating SQL, we should respect the original order of the selected columns and avoid reordering them to match the physical schema. While reordering can reveal more SELECT * optimizations when the query involves intermediate CTEs or subqueries, the current logic sorts the scan_list by the algebraic ordering of column names. This approach hides potential SELECT * optimizations, particularly in use cases like bpd.read_table("table_name")
There was a problem hiding this comment.
I still think this needs to be by physical schema id ideally, but I think it mostly doesn't matter when combined with other rewriters.
bigframes/core/nodes.py
Outdated
| @property | ||
| def is_star_selection(self) -> bool: | ||
| physical_names = tuple(item.name for item in self.source.table.physical_schema) | ||
| scan_names = tuple(item.source_id for item in self.scan_list.items) |
There was a problem hiding this comment.
the scan list item id must equal source_id as well for this to work as intended
There was a problem hiding this comment.
Good points. It should be addressed in the new commit. Thanks.
chelsea-lin
force-pushed
the
main_chelsealin_selectstart
branch
from
f5a2276 to
15dad78
Compare
chelsea-lin
changed the title
SELECT * optimization when compiling read-table nodes into sqlglotSELECT * optimization in the sqlglot compiler
chelsea-lin
force-pushed
the
main_chelsealin_selectstart
branch
from
15dad78 to
dd8825c
Compare
chelsea-lin
force-pushed
the
main_chelsealin_selectstart
branch
from
dd8825c to
a62598e
Compare
3 participants
Fixes internal issue 481740136 🦕