-
Notifications
You must be signed in to change notification settings - Fork 282
Expand file tree
/
Copy pathtemporal.py
More file actions
1165 lines (973 loc) · 56.6 KB
/
Copy pathtemporal.py
File metadata and controls
1165 lines (973 loc) · 56.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
"""Portable temporal value types for authored valid time (SPEC-82).
Basic Memory authors time as *semantic* data. A `[decision]` that was effective from
June 10 until the July 27 cutover is a statement about the world, not a record of when
the note was edited. This module owns the values that carry such a statement and the
lexical grammar for the range literals authors write.
PostgreSQL's range conventions are the language contract: `[lower,upper)` with explicit
inclusivity per side, unbounded ends, and a distinguished empty range. That is a
vocabulary choice, not a storage requirement -- these values reduce to portable scalars
so SQLite and Postgres can share one logical model. Its *discrete* canonicalization is
part of the contract too: a date range is stored as `[lower,upper)`, for the reason
`TemporalRange` documents. The author's own spelling is not lost -- it is kept verbatim
on `TemporalAssertion.source_text`.
Two canonical lexical forms carry every bound:
date ``YYYY-MM-DD`` (10 characters)
instant ``YYYY-MM-DDTHH:MM:SS.ffffffZ`` (27 characters, always UTC)
Both are fixed width with ASCII digits in fixed positions, so byte-lexicographic order
is chronological order. That is what lets containment and overlap be plain string
comparisons with identical SQL text in either dialect.
The two axes never mix and never convert into one another. A date bound is a calendar
date: it acquires no time of day and no timezone, ever. An instant bound names a moment
and is normalized to UTC, so two instants written in different offsets compare as the
instants they name. A timestamp written without an offset is *read as UTC*, which is
the convention the rest of the codebase already uses for naive datetimes
(`utils.ensure_timezone_aware`, `recent_activity`).
Two authored surfaces reach these values, and they trade precision for convenience in
opposite directions:
* A **range literal** (`[2026-06-10,2026-07-27)`) is the precise form. Its bounds must
be written in the canonical lexical shapes above, to at most microsecond precision.
* A **point** (`2026-06-10`, `2026-06`, `2026`, `June 10, 2026`) is the convenient form.
It denotes the span its precision covers, so an author never has to spell out a range to
say when something started. A point written in ISO calendar syntax is read literally,
because its text fixes its meaning; any other spelling is read with `dateparser`,
because there is no literal reading for a guess to contradict. Either way the reading
must be the same on every pass -- `yesterday` names no fixed span and is refused, for
the reasons set out above `parse_authored_point`.
"""
import re
from dataclasses import dataclass
from datetime import UTC, date, datetime, timedelta
from enum import StrEnum
from functools import lru_cache
from typing import TYPE_CHECKING, Any, Literal, assert_never, override
if TYPE_CHECKING: # pragma: no cover - import exists only for the annotation below
from dateparser.date import DateDataParser
class TemporalQualifierError(ValueError):
"""A temporal qualifier, range literal, or bound failed to parse or validate."""
class TimeKind(StrEnum):
"""Which kind of time an assertion describes.
`recorded` is deliberately absent: recorded time is never authored in markdown.
"""
EFFECTIVE = "effective"
VALID = "valid"
OCCURRED = "occurred"
DUE = "due"
MENTIONED = "mentioned"
class TemporalRangeAxis(StrEnum):
"""Whether a range is measured in calendar dates or in instants."""
DATE = "date"
INSTANT = "instant"
EMPTY_RANGE_LITERAL = "empty"
OBSERVATION_EXTRACTOR = "observation"
# Which component a slash-formatted date leads with. Only ambiguous forms consult it:
# `10/07/2026` is July 10 under YMD/DMY and October 7 under MDY, while `2026-06-10` is
# ISO and is never re-guessed. Mirrored by `BasicMemoryConfig.date_order`.
type DateOrder = Literal["YMD", "DMY", "MDY"]
DEFAULT_DATE_ORDER: DateOrder = "YMD"
# --- Bound grammar ---
# A date bound is exactly the canonical form, so authored and canonical text agree.
# The anchored pattern also rejects the compact `20260610` shape that
# `date.fromisoformat` accepts on 3.11+, which would break fixed-width ordering.
_DATE_BOUND = re.compile(r"^\d{4}-\d{2}-\d{2}$")
# Sub-microsecond precision is refused rather than truncated: silently dropping digits
# would make the stored bound name a different instant than the author wrote. The
# offset is optional because a naive timestamp is read as UTC, not rejected.
_INSTANT_BOUND = re.compile(
r"^\d{4}-\d{2}-\d{2}[Tt]\d{2}:\d{2}:\d{2}(?:\.\d{1,6})?(?:[Zz]|[+-]\d{2}:\d{2})?$"
)
_CANONICAL_INSTANT = re.compile(r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{6}Z$")
# Anything shaped like a date followed by a time separator is *meant* as a timestamp.
# Classifying it as an instant before validating it is what lets a broken timestamp
# report itself as one instead of as "not a calendar date".
_TIMESTAMP_SHAPE = re.compile(r"^\d{4}-\d{2}-\d{2}[Tt ]")
# `[lower,upper)` and friends. Bounds carry no brackets and no comma, so one anchored
# pattern splits the literal without any nesting rules.
_RANGE_LITERAL = re.compile(r"^([\[(])([^,\[\]()]*),([^,\[\]()]*)([\])])$")
def _classify_bound(bound: str) -> TemporalRangeAxis:
"""Decide which axis an authored bound is written on."""
if _TIMESTAMP_SHAPE.match(bound):
return TemporalRangeAxis.INSTANT
return TemporalRangeAxis.DATE
def _canonical_date(bound: str) -> str:
if not _DATE_BOUND.match(bound):
raise TemporalQualifierError(f"date bound must be YYYY-MM-DD: {bound!r}")
try:
return date.fromisoformat(bound).isoformat()
except ValueError as exc:
raise TemporalQualifierError(f"not a calendar date: {bound!r}") from exc
def _instant_value(moment: datetime) -> str | None:
"""Render one moment as the canonical fixed-width UTC instant.
A naive moment is read as UTC rather than refused. That is the house convention for
every other naive datetime in the codebase, and it is what lets an author write
`2026-07-27T18:42:00` without learning RFC 3339's offset syntax first.
None means the moment has no UTC rendering: shifting it by its offset carries it off
the calendar, as `9999-12-31T23:59:59-05:00` does into year 10000. Reported the way
`_next_calendar_day` reports its own edge -- each caller decides what running off the
calendar means for it -- rather than raised, so the overflow can never escape as a
bare `OverflowError` and fail a whole note's parse.
"""
if moment.tzinfo is None:
moment = moment.replace(tzinfo=UTC)
try:
utc = moment.astimezone(UTC)
except OverflowError:
return None
return utc.strftime("%Y-%m-%dT%H:%M:%S.%f") + "Z"
# The one component `datetime.fromisoformat` normalizes instead of refusing. It bounds an
# offset's *total* magnitude below 24 hours, so `+25:00` and `+9999` are rejected, but it
# does not bound the minutes field on its own: `+14:60` is carried into the hour and read as
# `+15:00`, and `+14:99` as `+15:39`. Both spellings have the hole -- `+1460` too -- and both
# signs. So the token names one instant and the stored bound names another, on every reindex.
#
# Minutes are the whole of it. Every other field is refused rather than normalized -- hour
# 25, minute 60 and second 60 of the time proper all raise -- and once minutes are held below
# 60 the total-magnitude check *is* RFC 3339's `00-23` on the hour, so the delegation that is
# right for the rest of the grammar stays right. This is the exception, not a lost trust.
_OVERFLOWING_OFFSET_MINUTES = re.compile(r"[+-]\d{2}:?[6-9]\d$")
def _iso_instant(text: str) -> datetime | None:
"""Parse one machine-syntax ISO timestamp, or None when its text names no moment.
The single place this module turns ISO text into a moment, shared by the range-literal
bounds and by authored points. They reach it through different grammars -- bounds are
held to the canonical RFC 3339 shape, an authored point to the wider set of machine
spellings people write -- but the validity question underneath is one question, and the
offset rule above only has to be stated once because of that.
RFC 3339 allows lowercase `t`/`z`, which `fromisoformat` rejects. Machine syntax carries
no other letters, so upper-casing only touches those two markers.
"""
if _OVERFLOWING_OFFSET_MINUTES.search(text):
return None
try:
return datetime.fromisoformat(text.upper())
except ValueError:
return None
def _canonical_instant(bound: str) -> str:
if not _INSTANT_BOUND.match(bound):
raise TemporalQualifierError(
f"timestamp bound must be RFC 3339 to microsecond precision, "
f"with an optional offset or Z: {bound!r}"
)
moment = _iso_instant(bound)
if moment is None:
raise TemporalQualifierError(f"not a valid timestamp: {bound!r}")
value = _instant_value(moment)
if value is None:
raise TemporalQualifierError(
f"timestamp bound leaves the calendar when converted to UTC: {bound!r}"
)
return value
def canonical_bound(bound: str, axis: TemporalRangeAxis) -> str:
"""Normalize one authored bound to the canonical fixed-width form for its axis."""
if axis is TemporalRangeAxis.DATE:
return _canonical_date(bound)
return _canonical_instant(bound)
def _require_canonical(value: str, axis: TemporalRangeAxis) -> None:
"""Reject a value that skipped `canonical_bound` on its way into a domain value."""
pattern = _DATE_BOUND if axis is TemporalRangeAxis.DATE else _CANONICAL_INSTANT
if not pattern.match(value):
raise TemporalQualifierError(f"{axis.value} bound is not canonical: {value!r}")
def _next_calendar_day(bound: str) -> str | None:
"""The canonical date after `bound`, or None when the calendar has none.
Only `9999-12-31` has no successor. Reporting that as None rather than raising lets
each side of a range decide what running off the end of the calendar means for it:
an upper end there covers every remaining day, a lower end past it covers none.
"""
day = date.fromisoformat(bound)
if day == date.max:
return None
return (day + timedelta(days=1)).isoformat()
# --- Values ---
@dataclass(frozen=True, slots=True)
class TemporalPoint:
"""One calendar date or instant that a containment question is asked about."""
axis: TemporalRangeAxis
value: str
def __post_init__(self) -> None:
_require_canonical(self.value, self.axis)
@override
def __str__(self) -> str:
return self.value
@dataclass(frozen=True, slots=True)
class TemporalRange:
"""One authored interval on a single time axis.
Bounds are canonical lexical strings; `None` means unbounded on that side.
Construction normalizes three PostgreSQL rules so no caller has to remember them:
an unbounded side is always exclusive, an interval containing no points *is* the
empty range, and -- exactly as `daterange` does -- a **date** range is rewritten
into the half-open `[lower,upper)` form.
That last rule is what makes the scalar SQL predicate correct rather than merely
tidy. Calendar dates are a *discrete* domain, so `[a,b]` and `[a,b+1)` denote the
same set of days, but only the half-open spelling lets endpoint comparisons decide
membership. Left as authored, `(2026-01-01,2026-01-03)` holds only January 2 and
`(2026-01-02,2026-01-04)` holds only January 3 -- disjoint sets -- yet each raw
endpoint lies inside the other's bounds, so a comparison of raw endpoints reports
an overlap that does not exist. Canonicalized to `[2026-01-02,2026-01-03)` and
`[2026-01-03,2026-01-04)`, the same comparison is right.
Instants are a continuous domain -- no moment is "the next one" -- so an instant
range keeps the inclusivity the author wrote and is never rewritten this way.
Canonicalization changes the *stored* spelling, never the set of times: `[a,a]`
becomes `[a,a+1)`, the one day `a`. What the author typed is not lost; it is kept
verbatim on `TemporalAssertion.source_text`, which is what serialization replays
and what a search result quotes back. `__str__` renders the canonical form, and
re-parsing that rendering yields this same value.
"""
axis: TemporalRangeAxis
lower: str | None = None
upper: str | None = None
lower_inclusive: bool = False
upper_inclusive: bool = False
is_empty: bool = False
def __post_init__(self) -> None:
if self.is_empty:
# The empty range has no endpoints at all, so inclusivity is meaningless
# for it; representing it two ways would make equality lie.
if (
self.lower is not None
or self.upper is not None
or self.lower_inclusive
or self.upper_inclusive
):
raise TemporalQualifierError("the empty range carries no bounds")
return
for bound in (self.lower, self.upper):
if bound is not None:
_require_canonical(bound, self.axis)
# Canonical bounds are fixed width, so string order is chronological order.
# Judged on the bounds as authored: an interval written backwards is an author
# error to report, not an empty range to accept silently.
if self.lower is not None and self.upper is not None and self.lower > self.upper:
raise TemporalQualifierError(
f"range lower bound {self.lower} is after upper bound {self.upper}"
)
# PostgreSQL: an unbounded side cannot be inclusive; there is no endpoint.
if self.lower is None:
object.__setattr__(self, "lower_inclusive", False)
if self.upper is None:
object.__setattr__(self, "upper_inclusive", False)
# --- Discrete canonical form ---
#
# Rewrite a date range to `[lower,upper)`. See the class docstring for why the
# scalar overlap predicate needs this and why instants must not get it.
if self.axis is TemporalRangeAxis.DATE:
if self.lower is not None and not self.lower_inclusive:
after_lower = _next_calendar_day(self.lower)
if after_lower is None:
# Nothing follows 9999-12-31, so a range starting strictly after it
# admits no date at all.
self._become_empty()
return
object.__setattr__(self, "lower", after_lower)
object.__setattr__(self, "lower_inclusive", True)
if self.upper is not None and self.upper_inclusive:
# None here loses no days: 9999-12-31 is the last date there is, so
# "through 9999-12-31 inclusive" and "unbounded above" hold the same
# set, and only the latter is representable in the canonical form.
object.__setattr__(self, "upper", _next_calendar_day(self.upper))
object.__setattr__(self, "upper_inclusive", False)
# PostgreSQL: an interval that admits no point at all *is* the empty range. The
# endpoints coincide without both being owned (`[a,a)`), or -- only reachable
# after the rewrite above, from `(a,a)` -- the lower end has overshot the upper.
if self.lower is not None and self.upper is not None:
admits_no_date = self.lower > self.upper or (
self.lower == self.upper and not (self.lower_inclusive and self.upper_inclusive)
)
if admits_no_date:
self._become_empty()
def _become_empty(self) -> None:
"""Collapse to the one empty representation, whatever bounds were written."""
object.__setattr__(self, "lower", None)
object.__setattr__(self, "upper", None)
object.__setattr__(self, "lower_inclusive", False)
object.__setattr__(self, "upper_inclusive", False)
object.__setattr__(self, "is_empty", True)
@classmethod
def empty(cls, axis: TemporalRangeAxis) -> "TemporalRange":
"""The empty range on one axis."""
return cls(axis=axis, is_empty=True)
@override
def __str__(self) -> str:
"""Render the canonical PostgreSQL range literal.
This is the normalized interval, not the author's text -- a date range always
renders half-open. Feeding the result back to `parse_range_literal` reproduces
this same value, so the rendering is a fixed point rather than a lossy view.
"""
if self.is_empty:
return EMPTY_RANGE_LITERAL
lower = "" if self.lower is None else self.lower
upper = "" if self.upper is None else self.upper
return (
f"{'[' if self.lower_inclusive else '('}{lower},{upper}"
f"{']' if self.upper_inclusive else ')'}"
)
@dataclass(frozen=True, slots=True)
class TemporalFilter:
"""A valid-time question asked of the stored assertions.
Exactly one of `at` (containment) or `overlaps` may be given, or neither -- a
kind-only filter asks for sources that carry *any* assertion of that kind, which
is a legal and useful question. A filter that asks nothing at all is refused
rather than silently matching everything.
"""
kind: TimeKind | None = None
at: TemporalPoint | None = None
overlaps: TemporalRange | None = None
def __post_init__(self) -> None:
if self.at is not None and self.overlaps is not None:
raise TemporalQualifierError(
"a temporal filter asks either 'at' or 'overlaps', never both"
)
if self.kind is None and self.at is None and self.overlaps is None:
raise TemporalQualifierError("a temporal filter must name a kind, a point, or a range")
@property
def window(self) -> TemporalRange | None:
"""The interval this filter tests against, or None for a kind-only filter.
Containment of a point is overlap with the closed range `[p,p]`: both ask
whether the stored interval and the queried interval share at least one point.
Collapsing them here lets one predicate answer both questions, which is also
why the two can never disagree about inclusivity or bounds. On the date axis
`TemporalRange` canonicalizes that window to `[p,p+1)` -- still the single day
`p`, now in the half-open form the predicate compares correctly.
"""
if self.at is not None:
return TemporalRange(
axis=self.at.axis,
lower=self.at.value,
upper=self.at.value,
lower_inclusive=True,
upper_inclusive=True,
)
return self.overlaps
@dataclass(frozen=True, slots=True)
class TemporalAssertion:
"""One authored statement that a source is valid over a span of time.
Source identity -- entity, source type, source row id -- is deliberately absent.
The parser reads markdown, where those ids do not exist yet; the projection layer
pairs this value with them when it writes derived rows.
`source_text` is the exact authored token. Serialization replays it verbatim, so a
parse/serialize round trip reproduces the author's bounds and precision even though
`valid_during` holds the normalized form.
"""
time_kind: TimeKind
valid_during: TemporalRange
source_text: str
extractor: str = OBSERVATION_EXTRACTOR
metadata: dict[str, Any] | None = None
# --- Literal parsing ---
def parse_range_literal(literal: str, *, axis: TemporalRangeAxis | None = None) -> TemporalRange:
"""Parse a PostgreSQL-style range literal into a canonical `TemporalRange`.
Accepts `[lower,upper)`, `(lower,upper]`, `[lower,)`, `(,upper)`, `(,)`, and the
bare token `empty`. `axis` asserts the axis the caller expects; when omitted it is
inferred from the bounds, which is why the bound-less forms require it explicitly.
"""
text = literal.strip()
if text == EMPTY_RANGE_LITERAL:
if axis is None:
raise TemporalQualifierError(
"the 'empty' range literal has no bounds, so its axis must be given"
)
return TemporalRange.empty(axis)
match = _RANGE_LITERAL.match(text)
if match is None:
raise TemporalQualifierError(
f"range literal must be [lower,upper), (lower,upper], or 'empty': {literal!r}"
)
open_bracket, lower_text, upper_text, close_bracket = match.groups()
lower_text = lower_text.strip()
upper_text = upper_text.strip()
written_axes = {_classify_bound(bound) for bound in (lower_text, upper_text) if bound}
if len(written_axes) > 1:
raise TemporalQualifierError(
f"a range must not mix date-only and timestamp bounds: {literal!r}"
)
if not written_axes:
if axis is None:
raise TemporalQualifierError(
f"a fully unbounded range has no bounds to classify: {literal!r}"
)
range_axis = axis
else:
range_axis = written_axes.pop()
if axis is not None and range_axis is not axis:
raise TemporalQualifierError(
f"expected {axis.value} bounds but found {range_axis.value} bounds: {literal!r}"
)
return TemporalRange(
axis=range_axis,
lower=canonical_bound(lower_text, range_axis) if lower_text else None,
upper=canonical_bound(upper_text, range_axis) if upper_text else None,
lower_inclusive=open_bracket == "[",
upper_inclusive=close_bracket == "]",
)
def parse_point(text: str) -> TemporalPoint:
"""Parse one authored date or timestamp into a canonical `TemporalPoint`."""
bound = text.strip()
if not bound:
raise TemporalQualifierError("a temporal point must not be empty")
axis = _classify_bound(bound)
return TemporalPoint(axis=axis, value=canonical_bound(bound, axis))
TEMPORAL_FILTER_FIELDS = ("valid_at", "valid_overlaps", "time_kind")
def reject_blank_temporal_value(field: str, value: str | None) -> None:
"""Refuse a valid-time field that is present but carries nothing.
`None` is how a caller says "no valid-time filter"; these fields are declared optional
precisely so that spelling exists. An empty or whitespace-only string is a different
statement -- a caller who believes they applied a filter -- and reading it as absence
is the failure this whole feature keeps having to close: a query that reports itself as
filtered, runs unfiltered, and answers with the undated rows the filter was meant to
exclude. Truthiness cannot tell the two apart, so presence is tested against `None`
everywhere on this path and blankness is refused here.
"""
if value is not None and not value.strip():
raise TemporalQualifierError(
f"{field} was given as an empty value; omit {field} to search without a "
f"valid-time filter"
)
def parse_temporal_filter(
*,
valid_at: str | None = None,
valid_overlaps: str | None = None,
time_kind: str | None = None,
) -> TemporalFilter | None:
"""Parse the three flat boundary fields into one portable filter value.
Every request surface -- HTTP, MCP, CLI -- carries a valid-time question as these
three independent strings, so this is the one place that turns them into the domain
value. Sharing it is what lets a caller validate the question *before* asking it and
be certain the answer to "is this filter well formed?" is the same one the search
service will reach.
Every rejection is deliberate and loud: an unknown kind, a malformed range literal, a
range mixing calendar dates with instants, or an impossible range raises rather than
degrading into a filter that quietly matches something else. A timestamp written
without an offset is not a rejection -- like every other naive datetime in the
codebase, it is read as UTC.
Returns None when no valid-time question was asked at all -- which means all three
fields are absent, not merely falsy. See `reject_blank_temporal_value`.
"""
for field, value in zip(TEMPORAL_FILTER_FIELDS, (valid_at, valid_overlaps, time_kind)):
reject_blank_temporal_value(field, value)
if valid_at is None and valid_overlaps is None and time_kind is None:
return None
kind: TimeKind | None = None
if time_kind is not None:
try:
kind = TimeKind(time_kind)
except ValueError as exc:
raise TemporalQualifierError(
f"unknown time_kind {time_kind!r}; expected one of "
f"{', '.join(item.value for item in TimeKind)}"
) from exc
return TemporalFilter(
kind=kind,
at=parse_point(valid_at) if valid_at is not None else None,
overlaps=parse_range_literal(valid_overlaps) if valid_overlaps is not None else None,
)
# --- Flexible authored points ---
@lru_cache(maxsize=16)
def _date_data_parser(date_order: DateOrder, relative_base: datetime) -> "DateDataParser":
"""The flexible reader for authored points, built once per order and reference instant.
Deferred import: dateparser costs ~0.13s and loads locale data, and the modules
that carry these values are imported on every CLI start (#886). Only an
observation that already looks like a qualifier ever reaches this function.
`relative_base` is the instant the reader treats as "now". It is always supplied
explicitly, never left to the wall clock, because the wall clock is what made a
reading depend on the day it ran -- see `_STABILITY_PROBE_BASES`.
"""
from dateparser.date import DateDataParser
return DateDataParser(
settings={
"DATE_ORDER": date_order,
# Makes `period` report "time" when the author wrote a clock reading,
# which is exactly the date-vs-instant distinction this module keeps.
"RETURN_TIME_AS_PERIOD": True,
# Fixes what "now" means for this reading, so relative wording resolves
# against a stated instant rather than the moment the indexer happened to run.
"RELATIVE_BASE": relative_base,
}
)
def _next_month_start(year: int, month: int) -> date | None:
"""The first day of the month after `year`-`month`, or None past the calendar's end.
Only December 9999 has no successor month; year 10000 is not a date `datetime` can
hold. Reported as None for the same reason `_next_calendar_day` reports its own
edge: the caller decides what running off the end of the calendar means for it.
"""
if month < 12:
return date(year, month + 1, 1)
if year == date.max.year:
return None
return date(year + 1, 1, 1)
def _calendar_span(lower: date, upper: date | None) -> TemporalRange:
"""The half-open calendar period `[lower,upper)`, unbounded when it runs to the end.
A period whose successor is off the calendar needs no upper end: nothing follows
9999-12-31, so `[lower,)` holds exactly the days `[lower,successor)` would have. It
is the same equivalence `TemporalRange` applies to an inclusive upper bound on the
last date, and it is why December 9999 is a period this reader can express rather
than one it fails on.
"""
return TemporalRange(
axis=TemporalRangeAxis.DATE,
lower=lower.isoformat(),
upper=None if upper is None else upper.isoformat(),
lower_inclusive=True,
)
# --- Which language an authored point is written in ---
#
# An author writes a point in one of two languages, and they come with opposite promises.
# **ISO calendar syntax** is machine syntax: the text fixes the meaning, so it must be read
# literally or refused. **Everything else** -- `June 10, 2026`, `2026/03/04`, `10/07/2026`
# -- is human syntax with no literal reading to contradict, so the flexible reader is
# trusted with it. Trusted to *read* it, that is: what it hands back must still name the
# same span whenever it is asked, which is what refuses `yesterday` further down.
#
# The variants below are what a point can be once that question is settled, and settling it
# *once* is the whole design. Four review rounds went the other way: each added a shape test
# whose failure meant "not my business", so a token that failed the test fell through to the
# flexible reader and the next round found another shape that failed it. Here the classifier
# is total: a token that opens with ISO syntax is an `_IsoDay`, an `_IsoMonth` or a
# `_MalformedIso`, and there is no fourth answer to fall through on.
#
# What that buys is narrower than "ISO-shaped text never reaches the flexible reader", and
# stating it precisely matters, because the loose version is false. An `_IsoDay`'s *trailing*
# text is still read by the flexible reader -- that is what reads `2026-06-10 10:00 AM`, and
# no grammar of clock spellings could. What the classifier settles for good is the *calendar*:
# a head that names no date dies here, and a real one is carried on the variant so the reading
# below can be held to it. The trailing is fenced by two rules instead, and dateparser's answer
# is believed only when both hold. It must come back as a time of day on the day the head names
# -- checked in `_read_iso_day`, against what it *returned*, since a suffix's looks do not say
# what it will do with it. And the text must not spell precision a canonical instant cannot
# carry -- checked here, on the text, because that is the one defect the returned-value check
# cannot see: a truncated fraction still lands on the right day.
# The ISO calendar components a point *opens* with: `YYYY-MM` and an optional `-DD`. A date
# carrying a time (`2026-06-10T14:00`, `2026-06-10 10:00 AM`) is matched on its date part
# alone, because `\d+` cannot cross the separator -- the rest is `trailing`, judged below.
#
# Each component is `\d+` rather than `\d{2}`, and nothing terminates the pattern, so the
# head matches whenever a point opens with ISO syntax at all. Both rules exist because the
# earlier cuts of this guard failed to match a malformed token and so let it escape: against
# `\d{2}` the day of `2026-01-0100` left a trailing `00` and matched nothing, and against a
# trailing `(?![\d-])` lookahead `2026-01-01-` matched nothing. Both reached the flexible
# reader, which is the one outcome ISO syntax must never have.
_ISO_CALENDAR_HEAD = re.compile(r"^(\d{4})-(\d+)(?:-(\d+))?")
# What it takes to be *reaching* for an ISO date, as opposed to naming one. A year and a
# hyphen is a commitment to machine syntax; nothing else is spelled that way. The head
# above still needs digits after that hyphen, so `2026--01`, `2026-` and `2026-x01` matched
# it not at all and fell to the flexible reader -- which invented January 2026, the whole of
# 2026, and *October 1st* respectively, none of which appears in the text. Claiming the
# opening separately is what makes the classifier total in the way it always claimed to be:
# a token that opens in ISO syntax is judged as ISO or refused, never handed on because the
# rest of it was too broken to parse. A bare `2026` carries no hyphen and is untouched.
_ISO_CALENDAR_OPENING = re.compile(r"^\d{4}-")
# --- Which language the *time* portion is written in ---
#
# The calendar portion has always been judged strictly: an ISO head either names a real date
# or the whole point is `_MalformedIso`, with no path to the lenient reader. The time portion
# never got that treatment. It was fenced instead by one returned-value check plus a text
# check per defect discovered -- one for over-long fractions, one for wrong-width calendar
# runs -- and a dangling `.` was simply the next defect no existing check named. Growing that
# list is the shape this module has been refactored away from twice.
#
# So the same question is asked of the trailing text that is asked of the head: which
# language is it in? **Letters mean a human spelling** -- `10:00 AM`, `2pm`, `14:00:00 UTC`,
# `at 14:00`, `noon` -- which has no literal reading for a guess to contradict, so the
# lenient reader is trusted with it exactly as the two-language contract requires. **Digits
# and punctuation alone mean machine syntax**, and machine syntax is read literally or
# refused. `T` and `Z` are the two exceptions: they are ISO's own markers rather than words.
#
# That split is what a shape test on `[T ]digit` alone cannot do. `10:00 AM`, `2pm` and
# `14:00:00 UTC` all open that way and all must stay lenient; every malformed spelling that
# reached the reader carried no letters at all.
_HUMAN_TIME_LETTER = re.compile(r"[^\W\dTtZz_]")
# The machine spellings of a time of day, as the trailing text after a complete ISO date.
# Seconds and the fraction are optional and the zone may be `Z`, `±HH:MM` or `±HHMM`, which
# is every ISO-time shape the pinned spellings use. A fraction must carry one to six digits,
# so this one grammar refuses both the over-long fraction and the dangling separator that
# needed a rule apiece before.
_ISO_TIME_TRAILING = re.compile(
r"^[Tt ]\d{2}:\d{2}(?::\d{2}(?:\.\d{1,6})?)?(?:[Zz]|[+-]\d{2}:?\d{2})?$"
)
def _is_machine_time(trailing: str) -> bool:
"""Whether trailing text after a date is written in machine syntax rather than words."""
return _HUMAN_TIME_LETTER.search(trailing) is None
def _named_calendar_date(year: str, month: str, day: str | None) -> date | None:
"""The date ISO-shaped calendar components name, or None when they name none.
A month-only head is placed on the first of that month: the day is a component the
author did not write, not one to guess at. `date` is the authority rather than a range
check because it already owns leap years and month lengths.
"""
# A month or a day is written with one or two digits, and that width is what separates
# an author's shorthand from an author's typo: `2026-1-5` is a legitimate unpadded
# spelling of a real date, while the `0100` in `2026-01-0100` is no day at all. Judged
# before `date`, which takes a C long and raises OverflowError -- not the ValueError
# below -- once a run of digits grows past it.
if len(month) > 2 or (day is not None and len(day) > 2):
return None
try:
return date(int(year), int(month), 1 if day is None else int(day))
except ValueError:
return None
@dataclass(frozen=True, slots=True)
class _IsoDay:
"""A point whose ISO head names a calendar day, and whatever was written after it.
The day is authoritative: it is what the author typed, so no reading of `trailing` may
contradict it. `trailing` is empty for a bare date; when it is not, the point is an
instant, because a time of day is the only thing that can follow a complete date.
"""
day: date
trailing: str
@dataclass(frozen=True, slots=True)
class _IsoMonth:
"""A point whose ISO head names a calendar month (`2026-06`), and so denotes it.
There is deliberately nowhere to put trailing text: nothing may follow a month. A clock
reading needs a day to fall on, and the flexible reader supplies the day it was not
given from *today*, so `2026-06 10:00` read as June 7 in March and June 1 in September
-- the same note projecting different valid time on different indexing days.
"""
year: int
month: int
@dataclass(frozen=True, slots=True)
class _MalformedIso:
"""A point written in ISO syntax that cannot be read as written.
`2026-13-01`, `2026-01-0100`, `2026-06 10:00`, `2026-01-01T10:00:00.1234567`. Either the
components name nothing on the calendar, or they name a moment finer than a canonical
instant records. The author reached for a machine date and missed, so there is no reading
to fall back on -- only a guess, which is what this variant exists to make unreachable.
"""
@dataclass(frozen=True, slots=True)
class _FlexiblePoint:
"""A point in no machine syntax at all, for the flexible reader to interpret."""
type _AuthoredPoint = _IsoDay | _IsoMonth | _MalformedIso | _FlexiblePoint
_MALFORMED_ISO = _MalformedIso()
_FLEXIBLE_POINT = _FlexiblePoint()
def _classify_authored_point(point: str) -> _AuthoredPoint:
"""Decide which language one authored point is written in, and what it names.
Total by construction, which is the property the whole design rests on: opening with
ISO syntax settles the question, and the three ISO variants are all a token can then
be. There is no "looks ISO but is not this function's business" answer to fall through
on, which is what every earlier cut of this guard offered and what each review round
found another way to reach.
"""
head = _ISO_CALENDAR_HEAD.match(point)
if head is None:
# Trigger: the token opens `YYYY-` but no calendar components could be read from it.
# Why: the author reached for a machine date and mistyped it. Handing that to the
# flexible reader is the one outcome ISO syntax must never have -- it does not
# report failure, it re-guesses, and a slipped keystroke becomes a confident date
# nobody wrote, reproduced identically by every reindex.
# Outcome: malformed, so the token stays observation content.
if _ISO_CALENDAR_OPENING.match(point):
return _MALFORMED_ISO
return _FLEXIBLE_POINT
year, month, day = head.groups()
named = _named_calendar_date(year, month, day)
if named is None:
return _MALFORMED_ISO
trailing = point[head.end() :]
if day is None:
# Trigger: the head names a month, with or without text after it.
# Why: a month is a complete point on its own, so anything following it is part of
# a date this head cannot carry -- see `_IsoMonth` for what reading it costs.
# Outcome: a bare month denotes its own period; a month with anything after it is
# malformed.
return _MALFORMED_ISO if trailing else _IsoMonth(int(year), int(month))
# Trigger: the text after the date spells a fraction of a second wider than six digits.
# Why: no reader here can store it, and the two that try disagree -- `_canonical_instant`
# refuses it, while the flexible reader truncates it and still answers with a time on
# the head's day, which is precisely what `_read_iso_day`'s returned-value check cannot
# catch. A guard that asks what came back cannot see digits that never made it in.
# Outcome: refused as malformed, so the strict and flexible paths give the same answer to
# the same text and the token stays observation content rather than a rounded instant.
# Trigger: text follows a complete date, written in machine syntax rather than words.
# Why: the head is already held to naming a real date; this holds the *time* to the same
# standard instead of leaving it to a returned-value check and a text rule per defect.
# A bare date has no trailing at all and is untouched; a worded clock is the other
# language and goes to the lenient reader, which is what the contract requires.
# Outcome: one grammar refuses every machine-syntax malformation -- the over-long
# fraction, the dangling separator, and the shapes nobody has written down yet.
if trailing and _is_machine_time(trailing) and not _ISO_TIME_TRAILING.match(trailing):
return _MALFORMED_ISO
return _IsoDay(named, trailing)
def _read_iso_day(
iso: _IsoDay, point: str, date_order: DateOrder, relative_base: datetime
) -> TemporalRange | None:
"""Read a point whose head names a calendar day, holding it to its own text."""
if not iso.trailing:
return TemporalRange(
axis=TemporalRangeAxis.DATE, lower=iso.day.isoformat(), lower_inclusive=True
)
if _is_machine_time(iso.trailing):
# Trigger: the time is written in machine syntax -- digits and punctuation, no words.
# Why: the classifier has already held its *shape* to `_ISO_TIME_TRAILING`, so what
# is left to establish is that those components name a real time. `fromisoformat`
# is the authority for that, exactly as `date` is for the calendar head: it knows
# hour 25 and minute 60 are not times, and it reads the shapes the grammar admits
# -- second-less, fractional, `Z`, `±HH:MM` and `±HHMM` alike. Upper-casing is safe
# because machine syntax carries no letters but ISO's own `t` and `z` markers.
# Outcome: an instant, or a refusal; never a guess, and never a rounded reading.
moment = _iso_instant(point)
if moment is None:
return None
instant = _instant_value(moment)
if instant is None:
# Shifting it to UTC carries it off the calendar, so it names no storable
# instant -- reported like any other unreadable point.
return None
return TemporalRange(axis=TemporalRangeAxis.INSTANT, lower=instant, lower_inclusive=True)
# The author wrote the clock in words, so the flexible reader
# is asked for it -- but only for it. What it hands back must be a time of day on the
# very day the head names, which is the check that keeps its guessing out of the answer:
# dateparser silently drops a suffix it cannot use (`2026-01-01T`, `2026-01-01Z`,
# `2026-01-01+14:00` all came back as the bare date), and a suffix it half-understands
# makes it abandon the ISO reading and re-guess the components under the configured
# order (`2026-06-10x` came back as October 6). Asking what it *returned* rather than
# what the suffix looks like is what covers every such shape, named or not.
date_data = _date_data_parser(date_order, relative_base).get_date_data(point)
moment = date_data.date_obj
if moment is None or date_data.period != "time" or moment.date() != iso.day:
return None
instant = _instant_value(moment)
if instant is None:
# A moment that leaves the calendar in UTC names no storable instant, so it reads
# as no date at all -- the token stays content.
return None
return TemporalRange(axis=TemporalRangeAxis.INSTANT, lower=instant, lower_inclusive=True)
# A fully numeric date, with the year written first or last. It is the one non-ISO shape
# whose meaning is *fixed* rather than guessed: `date_order` says which of the other two
# runs is the month, so the text plus one setting determine the date exactly and there is
# nothing left to interpret. dateparser does not treat it that way. Handed a run it cannot
# use where the author put it, it silently moves that run to the other slot and answers with
# a real date -- `2026/13/01` under YMD, or `13/01/2026` under MDY, both come back as
# January 13, a date the configured order does not name and the author did not write, refiled
# identically by every reindex. That is the `2026-13-01` disease in the one syntax the ISO
# classifier deliberately does not claim.
# The separator is `(\D)` matched twice rather than a list of the punctuation people
# use, because enumerating it is how this rule got half-applied the first time: written
# for `/` alone it left `2026.13.01`, `2026 13 01` and `2026_13_01` reaching the reader
# and coming back as January 13. What makes a token fully numeric is that its runs are
# digits and whatever stands between them does not, so that is what the pattern says.
_YEAR_FIRST_NUMERIC_DATE = re.compile(r"^(\d{4})(\D)(\d{1,2})\2(\d{1,2})$")
_YEAR_LAST_NUMERIC_DATE = re.compile(r"^(\d{1,2})(\D)(\d{1,2})\2(\d{4})$")
# Whether the first of the two non-year runs names the month, given the configured order and
# where the year was written. Five of the six are the order read literally. The sixth is not:
# `YMD` describes no arrangement that ends in the year, so with the year last the reader
# falls back to day-first, and that fallback is this module's behaviour too -- pinned by the
# `10/07/2026` cases rather than left implicit. Naming all six is what lets the check compare
# against a reading it can state, instead of trusting whatever came back.
_MONTH_LEADS_THE_REMAINDER: dict[tuple[DateOrder, bool], bool] = {
("YMD", True): True,
("MDY", True): True,
("DMY", True): False,
("MDY", False): True,
("DMY", False): False,
("YMD", False): False,
}
def _ordered_numeric_date(point: str, date_order: DateOrder) -> date | None | Literal[False]:
"""The date a fully numeric token names under `date_order`.
`False` means the token is not that shape and this rule has nothing to say about it;
`None` means it is, and names no date on the calendar.
A two-digit year is not this shape: `03/04/26` leaves which run is even the year to the
reader, so there is no stated reading to hold it to.
"""
year_first = _YEAR_FIRST_NUMERIC_DATE.match(point)
ordered = year_first or _YEAR_LAST_NUMERIC_DATE.match(point)
if ordered is None:
return False
# groups() is (run, separator, run, run); the separator is captured only so the
# backreference can require the same one twice.
year_run, _separator, second_run, third_run = ordered.groups()
runs = [int(year_run), int(second_run), int(third_run)]
year = runs.pop(0) if year_first else runs.pop()
first, second = runs
month, day = (
(first, second)
if _MONTH_LEADS_THE_REMAINDER[(date_order, bool(year_first))]
else (second, first)
)
try:
return date(year, month, day)
except ValueError:
return None
def _read_flexible_point(
point: str, date_order: DateOrder, relative_base: datetime
) -> TemporalRange | None:
"""Read a point written in no machine syntax, taking the flexible reader at its word."""
date_data = _date_data_parser(date_order, relative_base).get_date_data(point)
moment = date_data.date_obj
if moment is None:
return None
# Trigger: a year-first numeric date, whose meaning `date_order` fixes exactly.
# Why: the reader is free to move a run it cannot use where the author put it, and
# `2026/13/01` under YMD comes back as January 13 rather than as the impossible month
# the author actually typed. Holding it to the order is the same rule the ISO
# classifier applies to `2026-13-01`, in the syntax that classifier does not claim.
# Outcome: refused when the order names no date, and when the reader answered with a
# different one than the order names -- never quietly re-ordered.
ordered = _ordered_numeric_date(point, date_order)
if ordered is not False and moment.date() != ordered:
return None
# dateparser fills components the author did not write from the reference instant, so
# only the components `period` vouches for may be read off `moment`. Discarding the
# rest is also what lets `June 2026` survive the stability check: the filled-in day
# differs between probes, and the month this builds from it does not.
match date_data.period:
case "time":
instant = _instant_value(moment)
if instant is None:
# A moment that leaves the calendar in UTC names no storable instant,
# so it reads as no date at all -- the token stays content.
return None
return TemporalRange(
axis=TemporalRangeAxis.INSTANT,
lower=instant,
lower_inclusive=True,
)
case "year":
# The month after December is the following January 1 -- except at year
# 9999, where there is none and `_calendar_span` leaves the span open at
# `[9999-01-01,)`, which is still exactly that year.
return _calendar_span(date(moment.year, 1, 1), _next_month_start(moment.year, 12))
case "month":
return _calendar_span(
date(moment.year, moment.month, 1),
_next_month_start(moment.year, moment.month),
)