Skip to content

Fix date_bin_gapfill end time with two inclusive upper bounds on time - #19249

Open
PDGGK wants to merge 1 commit into
apache:masterfrom
PDGGK:fix-gapfill-two-upper-bounds
Open

PDGGK wants to merge 1 commit into
apache:masterfrom
PDGGK:fix-gapfill-two-upper-bounds

Conversation

@PDGGK

@PDGGK PDGGK commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Description

When the WHERE clause of a date_bin_gapfill query has two inclusive upper bounds on time, the gap fill ends at the larger one. It then outputs windows after the range that the WHERE clause selects:

SELECT date_bin_gapfill(1h, time) AS hour_time, avg(s1) AS avg_s1 FROM table1
WHERE time BETWEEN 2024-09-24T04:00:00+00:00 AND 2024-09-24T13:00:00+00:00
  AND time <= 2024-09-24T10:00:00+00:00 AND device_id = 'd1'
GROUP BY 1

This returns windows up to 13:00, although no row after 10:00 can match. The same happens with time <= a AND time <= b in either order.

GapFillStartAndEndTimeExtractVisitor.Context.updateEndTime keeps the tighter bound in every other case: the smaller of two < bounds, and the right one when < and <= are mixed. For two <= bounds it used Math.max (GapFillStartAndEndTimeExtractVisitor.java:278).

Change

Use Math.min for two <= bounds.

#18590, a draft, adds a test class with the same name for the Long boundary cases. None of its tests reaches this line, so if both go in, the second one only needs the two classes merged into one file.

Verification

  • Unit test. New GapFillStartAndEndTimeExtractVisitorTest checks the inferred range for two upper bounds in both orders, BETWEEN with <=, mixed < and <=, and the same cases for lower bounds. On master 2cd3b07, testTighterUpperBoundIsUsed fails with expected:<5> but was:<10> and passes with the change. The lower-bound test passes in both cases.
  • IT. New twoUpperBoundsGapFillTest in IoTDBGapFillTableIT runs the query above, and the same query with time >= ... AND time <= 13:00 AND time <= 10:00. IoTDBGapFillTableIT passes with -PTableSimpleIT and -PTableClusterIT (1C3D). Without the change the new test fails at the first query, which returns more than the seven expected rows.
  • Real server. On apache/iotdb:2.0.11-standalone, which has the same line, both queries return ten windows on the IT's data, from 04:00 to 13:00. With the changed visitor class in iotdb-server-2.0.11.jar they return the seven expected rows.
  • spotless:check passes for datanode and integration-test.

This PR has:

  • been self-reviewed.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage.
  • added integration tests.

Key changed/added classes (or packages if there are too many classes) in this PR
  • GapFillStartAndEndTimeExtractVisitor
  • IoTDBGapFillTableIT, GapFillStartAndEndTimeExtractVisitorTest

When the WHERE clause has two inclusive upper bounds on time, such as
time <= 10 AND time <= 5 or time BETWEEN 1 AND 10 AND time <= 5, the gap
fill took the larger one as its end time. It then output windows after
the end of the range that the WHERE clause selects. Take the smaller one,
as is already done for two exclusive upper bounds.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant