Add more type operator implementations for flat representations #27196

pettyjamesm · 2025-11-03T17:25:25Z

Description

Adds equals operator implementations for various fixed width types to directly check block position vs flat representations for equality. Also adds xxhash64 operator implementations that directly hash the flat representation as well. Previously, these types had synthesized equals and xxhash operator implementations that would read the flat and block implementations into on-stack representations and compare (or hash) those which often involved extra ephemeral object creations.

Also adds a slew of parameter annotations for operator methods working on flat types. The missing annotations caused no issue, but adding them clarifies the intent and usage.

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

## Section
* Fix some things. ({issue}`issuenumber`)

Summary by Sourcery

Add direct equals operator implementations for fixed-width SPI types and annotate flat-type operator parameters for clearer intent

Enhancements:

Implement ScalarOperator(EQUAL) methods for various fixed-width types to compare flat and block representations without creating temporary objects
Annotate flat-write and flat-read operator method parameters with @FlatFixed, @FlatFixedOffset, @FlatVariableWidth, and @FlatVariableOffset across SPI types for improved clarity

Summary by Sourcery

Implement direct scalar operators for fixed-width SPI types to compare and hash flat and block representations without creating temporary objects, and add flat-type parameter annotations for clearer intent.

New Features:

Add direct EQUAL, HASH_CODE, and XX_HASH_64 scalar operators for various fixed-width SPI types to compare and hash flat vs block representations without on-stack object creation

Enhancements:

Annotate flat operator method parameters with @FlatFixed, @FlatFixedOffset, @FlatVariableWidth, and @FlatVariableOffset across SPI types for improved parameter clarity

sourcery-ai · 2025-11-03T17:25:33Z

Reviewer's Guide

This PR introduces direct flat-slice operator implementations for EQUAL, HASH_CODE, and XX_HASH_64 across fixed-width SPI types to avoid intermediate object allocations, and adds explicit @Flat* annotations on flat operator parameters for clearer intent and correctness.

Class diagram for updated operator methods with flat-slice support and annotations

classDiagram
    class LongTimeWithTimeZoneType {
        +writeFlat(LongTimeWithTimeZone value, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
        +writeBlockFlat(@BlockPosition Fixed12Block block, @BlockIndex int position, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
        +equalOperator(@FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset, @BlockPosition Fixed12Block rightBlock, @BlockIndex int rightPosition)
        +hashCodeOperator(@FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
        +xxHash64Operator(@FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
    }
    class LongTimestampType {
        +writeFlat(LongTimestamp value, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
        +writeBlockFlat(@BlockPosition Fixed12Block block, @BlockIndex int position, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
        +equalOperator(@FlatFixed byte[] leftFixedSizeSlice, @FlatFixedOffset int leftFixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset, @BlockPosition Fixed12Block rightBlock, @BlockIndex int rightPosition)
        +xxHash64Operator(@FlatFixed byte[] leftFixedSizeSlice, @FlatFixedOffset int leftFixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
    }
    class LongTimestampWithTimeZoneType {
        +writeFlat(LongTimestampWithTimeZone value, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
        +writeBlockFlat(@BlockPosition Fixed12Block block, @BlockIndex int position, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
        +equalOperator(@FlatFixed byte[] leftFixedSizeSlice, @FlatFixedOffset int leftFixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset, @BlockPosition Fixed12Block rightBlock, @BlockIndex int rightPosition)
        +xxHash64Operator(@FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
    }
    class LongDecimalType {
        +writeFlat(Int128 value, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
        +writeBlockToFlat(@BlockPosition Int128ArrayBlock block, @BlockIndex int position, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
        +equalOperator(@FlatFixed byte[] leftFixedSizeSlice, @FlatFixedOffset int leftFixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset, @BlockPosition Int128ArrayBlock rightBlock, @BlockIndex int rightPosition)
        +xxHash64Operator(@FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
    }
    class IpAddressType {
        +writeFlat(Slice value, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
        +equalOperator(@FlatFixed byte[] leftFixedSizeSlice, @FlatFixedOffset int leftFixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset, @BlockPosition Int128ArrayBlock rightBlock, @BlockIndex int rightPosition)
        +xxHash64Operator(@FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
    }
    class UuidType {
        +writeFlat(Slice sourceSlice, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
        +equalOperator(@FlatFixed byte[] leftFixedSizeSlice, @FlatFixedOffset int leftFixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset, @BlockPosition Int128ArrayBlock rightBlock, @BlockIndex int rightPosition)
        +xxHash64Operator(@FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
    }
    class ShortDecimalType {
        +writeFlat(long value, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
        +equalOperator(@FlatFixed byte[] leftFixedSizeSlice, @FlatFixedOffset int leftFixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset, @BlockPosition LongArrayBlock rightBlock, @BlockIndex int rightPosition)
    }
    class AbstractIntType {
        +writeFlat(long value, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
    }
    class AbstractLongType {
        +writeFlat(long value, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
    }
    class BooleanType {
        +writeFlat(boolean value, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
    }
    class DoubleType {
        +writeFlat(double value, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
    }
    class RealType {
        +writeFlat(long value, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
    }
    class ShortTimeWithTimeZoneType {
        +writeFlat(long value, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
    }
    class ShortTimestampType {
        +writeFlat(long value, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
    }
    class TimeType {
        +writeFlat(long value, @FlatFixed byte[] fixedSizeSlice, @FlatFixedOffset int fixedSizeOffset, @FlatVariableWidth byte[] unusedVariableSizeSlice, @FlatVariableOffset int unusedVariableSizeOffset)
    }

File-Level Changes

Change	Details	Files
Add direct flat operator implementations for equals, hashCode, and xxHash64 to bypass temporary object creation	Introduce @ScalarOperator(EQUAL) methods that compare values by reading directly from flat slices without constructing on-stack objects Introduce @ScalarOperator(HASH_CODE) methods to compute hash codes directly from flat slices Introduce @ScalarOperator(XX_HASH_64) methods to compute xxhash64 values directly from flat slices	`core/trino-spi/src/main/java/io/trino/spi/type/LongTimeWithTimeZoneType.java` `core/trino-spi/src/main/java/io/trino/spi/type/LongTimestampType.java` `core/trino-spi/src/main/java/io/trino/spi/type/LongTimestampWithTimeZoneType.java` `core/trino-spi/src/main/java/io/trino/spi/type/LongDecimalType.java` `core/trino-main/src/main/java/io/trino/type/IpAddressType.java` `core/trino-spi/src/main/java/io/trino/spi/type/UuidType.java` `core/trino-spi/src/main/java/io/trino/spi/type/ShortDecimalType.java`
Annotate flat operator method parameters with dedicated @Flat* annotations for clarity	Add @FlatFixed and @FlatFixedOffset on byte[] and offset parameters Add @FlatVariableWidth and @FlatVariableOffset on variable slice parameters	`core/trino-spi/src/main/java/io/trino/spi/type/LongTimeWithTimeZoneType.java` `core/trino-spi/src/main/java/io/trino/spi/type/LongTimestampType.java` `core/trino-spi/src/main/java/io/trino/spi/type/LongTimestampWithTimeZoneType.java` `core/trino-spi/src/main/java/io/trino/spi/type/LongDecimalType.java` `core/trino-main/src/main/java/io/trino/type/IpAddressType.java` `core/trino-spi/src/main/java/io/trino/spi/type/UuidType.java` `core/trino-spi/src/main/java/io/trino/spi/type/ShortDecimalType.java` `core/trino-spi/src/main/java/io/trino/spi/type/AbstractIntType.java` `core/trino-spi/src/main/java/io/trino/spi/type/AbstractLongType.java` `core/trino-spi/src/main/java/io/trino/spi/type/BooleanType.java` `core/trino-spi/src/main/java/io/trino/spi/type/DoubleType.java` `core/trino-spi/src/main/java/io/trino/spi/type/RealType.java` `core/trino-spi/src/main/java/io/trino/spi/type/ShortTimeWithTimeZoneType.java` `core/trino-spi/src/main/java/io/trino/spi/type/ShortTimestampType.java` `core/trino-spi/src/main/java/io/trino/spi/type/TimeType.java`

Tips and commands

Interacting with Sourcery

Trigger a new review: Comment @sourcery-ai review on the pull request.
Continue discussions: Reply directly to Sourcery's review comments.
Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it. You can also reply to a
review comment with @sourcery-ai issue to create an issue from it.
Generate a pull request title: Write @sourcery-ai anywhere in the pull
request title to generate a title at any time. You can also comment
@sourcery-ai title on the pull request to (re-)generate the title at any time.
Generate a pull request summary: Write @sourcery-ai summary anywhere in
the pull request body to generate a PR summary at any time exactly where you
want it. You can also comment @sourcery-ai summary on the pull request to
(re-)generate the summary at any time.
Generate reviewer's guide: Comment @sourcery-ai guide on the pull
request to (re-)generate the reviewer's guide at any time.
Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
pull request to resolve all Sourcery comments. Useful if you've already
addressed all the comments and don't want to see them anymore.
Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
request to dismiss all existing Sourcery reviews. Especially useful if you
want to start fresh with a new review - don't forget to comment
@sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

Enable or disable review features such as the Sourcery-generated pull request
summary, the reviewer's guide, and others.
Change the review language.
Add, remove or edit custom review instructions.
Adjust other review settings.

Getting Help

Contact our support team for questions or feedback.
Visit our documentation for detailed guides and information.
Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

sourcery-ai

Hey there - I've reviewed your changes - here's some feedback:

There’s a lot of repeated flat-block operator boilerplate across types—consider refactoring common logic into a shared helper or code generator to reduce duplication and maintenance overhead.
Double-check that each fixed-width type has a consistent set of EQUAL, HASH_CODE, and XX_HASH_64 flat implementations so you don’t end up with gaps in operator support.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- There’s a lot of repeated flat-block operator boilerplate across types—consider refactoring common logic into a shared helper or code generator to reduce duplication and maintenance overhead.
- Double-check that each fixed-width type has a consistent set of EQUAL, HASH_CODE, and XX_HASH_64 flat implementations so you don’t end up with gaps in operator support.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

starburstdata-automation · 2025-11-04T15:54:49Z

Started benchmark workflow for this PR with test type = iceberg/sf1000_parquet_part.

Building Trino finished with status: success
Benchmark finished with status: success
Comparing results to the static baseline values, follow above workflow link for more details/logs.
Status message: NO Regression found.
Benchmark Comparison to the closest run from Master: Report

wendigo · 2025-11-04T15:55:02Z

@pettyjamesm running benchmarks now :)

starburstdata-automation · 2025-11-04T17:16:08Z

Started benchmark workflow for this PR with test type = iceberg/sf1000_parquet_unpart.

Building Trino finished with status: success
Benchmark finished with status: success
Comparing results to the static baseline values, follow above workflow link for more details/logs.
Status message: NO Regression found.
Benchmark Comparison to the closest run from Master: Report

cla-bot bot added the cla-signed label Nov 3, 2025

pettyjamesm force-pushed the add-fixed-flat-equals branch from 49f7b91 to 091c4e8 Compare November 3, 2025 17:25

pettyjamesm added 15 commits November 3, 2025 13:01

Add missing annotations to LongDecmialType operators

b22684e

Add missing annotations to ShortDecimalType operators

ca30594

Add missing annotations to LongTimeWithTimeZoneType

26d6205

Add missing annotations to LongTimestampWithTimeZoneType

9aa8dbe

Add missing annotations to LongTimestampType

4d5a506

Add missing annotations to ShortTimeWithTimeZoneType

730f330

Add missing annotations to TimeType

3bac844

Add missing annotations to BooleanType

c267eb4

Add missing annotations to AbstractIntType

f26e628

Add missing annotations to AbstractLongType

9514375

Add missing annotations to DoubleType

89603d1

Add missing annnotations to IpAddressType

2c91a92

Add missing annotations to UuidType

9fc978a

Add missing annotations to RealType

ca2dcef

Add missing annotations to ShortTimestampType

571e406

pettyjamesm force-pushed the add-fixed-flat-equals branch from 091c4e8 to 2ae4c41 Compare November 3, 2025 18:17

pettyjamesm requested a review from raunaqmorarka November 3, 2025 18:26

pettyjamesm marked this pull request as ready for review November 3, 2025 18:34

sourcery-ai bot reviewed Nov 3, 2025

View reviewed changes

pettyjamesm changed the title ~~Add more equals operator implementations~~ Add more type operator implementations for flat representation Nov 3, 2025

pettyjamesm changed the title ~~Add more type operator implementations for flat representation~~ Add more type operator implementations for flat representations Nov 3, 2025

pettyjamesm added 4 commits November 3, 2025 13:46

Add missing annotations to AbstractVariableWidthType

b073fce

Add flat equalsOperator implementations

ce431fa

Add flat xxhash64 operator implementations

7122378

Add flat hashcode implementation for LongTimeWithTimeZoneType

c273385

pettyjamesm force-pushed the add-fixed-flat-equals branch from 8b7b781 to c273385 Compare November 3, 2025 18:47

pettyjamesm requested a review from dain November 3, 2025 21:06

pettyjamesm requested a review from wendigo November 4, 2025 15:12

trinodb deleted a comment from starburstdata-automation Nov 4, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add more type operator implementations for flat representations #27196

Add more type operator implementations for flat representations #27196

pettyjamesm commented Nov 3, 2025 •

edited by sourcery-ai bot

Loading

Uh oh!

sourcery-ai bot commented Nov 3, 2025 •

edited

Loading

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

sourcery-ai bot left a comment

Uh oh!

starburstdata-automation commented Nov 4, 2025 •

edited

Loading

Uh oh!

wendigo commented Nov 4, 2025

Uh oh!

starburstdata-automation commented Nov 4, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

Add more type operator implementations for flat representations #27196

Are you sure you want to change the base?

Add more type operator implementations for flat representations #27196

Conversation

pettyjamesm commented Nov 3, 2025 • edited by sourcery-ai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Release notes

Summary by Sourcery

Summary by Sourcery

Uh oh!

sourcery-ai bot commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviewer's Guide

Class diagram for updated operator methods with flat-slice support and annotations

File-Level Changes

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

sourcery-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

starburstdata-automation commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wendigo commented Nov 4, 2025

Uh oh!

starburstdata-automation commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

pettyjamesm commented Nov 3, 2025 •

edited by sourcery-ai bot

Loading

sourcery-ai bot commented Nov 3, 2025 •

edited

Loading

starburstdata-automation commented Nov 4, 2025 •

edited

Loading

starburstdata-automation commented Nov 4, 2025 •

edited

Loading