Repository navigation
Fail a retried upsert instead of inserting a concurrently inserted key - #4392
Anurag-S1ngh wants to merge 2 commits into
Conversation
A commit retry replays the append that was planned before the concurrent commit, and the fast append producer skips concurrency validation, so a key another writer inserted after the upsert read ends up inserted twice. Record the filter for the rows an upsert inserts on the append producer and validate the files added since the starting snapshot against it when the commit is rebuilt for a retry. The upsert now fails with a CommitFailedException instead of leaving a duplicate key.
Fokko
left a comment
There was a problem hiding this comment.
Thanks for picking this up, the approach of validating the added data files against the inserted keys on retry looks right to me. A few comments inline.
| self._upsert_insert_filter = insert_filter | ||
| self._case_sensitive = case_sensitive | ||
|
|
||
| def _validate_concurrency(self) -> None: |
There was a problem hiding this comment.
This check runs regardless of the table's isolation level. The update half of the upsert only runs _validate_added_data_files under serializable (see the base _validate_concurrency), so with write.update.isolation-level=snapshot the two halves of a single upsert behave differently: the update half tolerates concurrently added files, while the insert half rejects them. This matches Java, where MERGE only validates conflicting data under serializable isolation. Should we check isolation_level(Operation.OVERWRITE) here so both halves follow the same setting?
| self._case_sensitive, | ||
| ) | ||
| except ValidationException as e: | ||
| raise CommitFailedException( |
There was a problem hiding this comment.
A conflict on the update half of the upsert raises ValidationException, but here we convert it into a CommitFailedException. This means that the same upsert raises a different exception depending on which half conflicted. I can see the argument for CommitFailedException (it's what 0.11.1 raised, and the caller can simply retry the whole upsert), but I think we should be consistent across both halves. What do you think about letting the ValidationException propagate?
| df: pa.Table | pa.RecordBatchReader, | ||
| snapshot_properties: dict[str, str] = EMPTY_DICT, | ||
| branch: str | None = MAIN_BRANCH, | ||
| _upsert_insert_filter: BooleanExpression | None = None, |
There was a problem hiding this comment.
I'm not a big fan of adding private _upsert_insert_filter / _case_sensitive arguments to the public Transaction.append signature, since they show up in the API docs. Could upsert configure the producer directly instead? Also, the concept is not specific to upsert; it's a conflict detection filter. Something like _conflict_detection_filter / validate_no_conflicting_data(filter) would line up with Java's OverwriteFiles/RowDelta. Last, set_upsert_insert_filter also overwrites the inherited _case_sensitive, which is harmless today but easy to trip over later.
Address the review feedback on the upsert retry check: - Gate the check on write.update.isolation-level, so both halves of an upsert follow the same isolation setting. - Let the ValidationException propagate instead of converting it to a CommitFailedException, matching the update half. - Drop the private Transaction.append arguments and let upsert configure the append producer through validate_no_conflicting_data. The filter's case sensitivity is stored on the producer instead of overwriting its inherited value.
Fokko
left a comment
There was a problem hiding this comment.
Thanks @Anurag-S1ngh, nice catch and a clean fix! The second commit addressing the isolation level gating and letting the ValidationException propagate reads well.
I pulled the branch and ran tests/table/test_upsert.py locally, all 29 pass, and CI is green.
A couple of small things, none blocking:
pyiceberg/table/update/snapshot.py:695:validate_no_conflicting_dataonly stores the filter, the actual check runs in_validate_concurrencyon retry. Maybe worth a docstring line saying the validation happens at commit/retry time, so callers don't expect an exception from the call itself.pyiceberg/table/update/snapshot.py:693: the class level defaults on_FastAppendFilesmean a producer that never callsvalidate_no_conflicting_datakeeps skipping validation, which is what plain appends want. Good.- The snapshot isolation test documenting that the duplicate can still land under
write.update.isolation-level=snapshotis a nice touch, that makes the semantics explicit.
LGTM once CI stays green!
Summary
A commit retry replays the append planned before the concurrent commit. The fast append producer skips concurrency validation, so a key another writer inserted after the upsert read was inserted a second time.
Changes
CommitFailedExceptioninstead of inserting a duplicate key.Testing
make testpasses (4697 passed, 5 skipped).Closes #4194