feat: Added SQL registry schema_mode and registry create command - #6704
Conversation
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6704 +/- ##
==========================================
+ Coverage 46.75% 46.76% +0.01%
==========================================
Files 414 415 +1
Lines 50348 50392 +44
Branches 7208 7214 +6
==========================================
+ Hits 23539 23568 +29
- Misses 25157 25171 +14
- Partials 1652 1653 +1
Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
|
@AdityaPatil22 Thanks for this — it's a focused, backward-compatible change that fills a real gap (running the SQL registry against a database where the runtime role shouldn't hold DDL grants). Overall looks good; a few minor points below, none blocking. Strengths
Suggestions (minor / follow-up)
Nits
|
|
@nquinn408 Thanks for the thorough review! Addressed all points: verify skips the read replica - _verify_schema now runs against read_engine too when it differs from write_engine, mirroring the existing _warn_if_narrow_blob_columns pattern. teardown() still issues DDL - On closer inspection, teardown() actually runs DELETE FROM (DML), not DROP TABLE (DDL), so it doesn't need DDL privileges. Updated the doc to correct this pre-existing inaccuracy: "the registry deletes all rows from the registry tables (it does not drop the tables themselves). This runs regardless of schema_mode and requires only DML (DELETE) privileges, not DDL." test_schema_mode_skip_does_not_run_ddl - Rewritten to mock metadata.create_all and SqlRegistry._verify_schema, asserting neither is called. This directly proves the "no DDL, no verification" claim — the old test pre-created the schema so it couldn't distinguish skip from auto/verify. Nits:
|
There was a problem hiding this comment.
One residual consideration (non-blocking)
Adding _verify_schema(self.read_engine) means verify mode now hard-fails startup if the read replica is missing a table — even transiently. There's already a comment nearby noting "a read replica can be on a different schema version (e.g. mid-migration)." For verify that strictness is arguably the intent (you asked it to assert the schema), so I'd keep it — but it's worth the author being aware that a lagging replica mid-migration will now block startup in verify mode. A one-line doc mention would be nice; not required.
Signed-off-by: Aditya Patil <adityapatil7649@gmail.com>
Signed-off-by: Aditya Patil <adityapatil7649@gmail.com>
Signed-off-by: Aditya Patil <adityapatil7649@gmail.com>
1f64eee to
d505e03
Compare
|
@nquinn408 - I have updated the doc as suggested |
|
@AdityaPatil22 Where are we with this PR? There is a test failure. Thanks for your work on this! |
What this PR does / why we need it:
Adds a schema_mode configuration option to the SQL registry that controls whether Feast creates database tables on startup. This addresses production environments where the application database user should not have DDL (CREATE TABLE) privileges.
Three modes are supported:
Also adds a feast registry create-schema CLI command to pre-create the schema with a privileged user, so the runtime user only needs DML access.
Which issue(s) this PR fixes:
Fixes #6678
Checks
Testing Strategy
Unit tests - 5 new tests in TestSchemaMode covering: auto creates tables, verify raises on missing tables, verify passes with pre-created schema, skip works with existing schema, invalid values rejected by Pydantic.
Manual tests - Verified all three modes end-to-end against SQLite: auto creates and applies, verify succeeds with existing schema and fails without, skip works, east registry create-schema creates schema for verify/skip to use.