fix: Ensure Postgres queries are committed or autocommit is used #5039
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: TomSteenbergen <tomsteenbergen1995@gmail.com>
Signed-off-by: TomSteenbergen <tomsteenbergen1995@gmail.com>
Signed-off-by: TomSteenbergen <tomsteenbergen1995@gmail.com>
franciscojavierarceo
approved these changes
Feb 11, 2025
Member
franciscojavierarceo
left a comment
franciscojavierarceo
left a comment
There was a problem hiding this comment.
Thank you for this!
franciscojavierarceo
pushed a commit
that referenced
this pull request
Feb 17, 2025
# [0.46.0](v0.45.0...v0.46.0) (2025-02-17) ### Bug Fixes * Add scylladb to online stores list in docs ([#5061](#5061)) ([08183ed](08183ed)) * Changed feast operator to set status of featurestore cr to ready based on deployment.status = available ([#5020](#5020)) ([fce0d35](fce0d35)) * Ensure Postgres queries are committed or autocommit is used ([#5039](#5039)) ([46f8d7a](46f8d7a)) * Fixing the release workflow to refresh the stable branch when the release is not running in the dry run mode. ([#5057](#5057)) ([a13fa9b](a13fa9b)) * Operator - make onlineStore the default service ([#5044](#5044)) ([6c92447](6c92447)) * Operator - resolve infinite reconciler loop in authz controller ([#5056](#5056)) ([11e4548](11e4548)) * Resolve module on windows ([#4827](#4827)) ([efbffa4](efbffa4)) * Setting the github_token explicitly to see if that solves the problem. ([#5012](#5012)) ([3834ffa](3834ffa)) * Validate entities when running get_online_features ([#5031](#5031)) ([3bb0dca](3bb0dca)) ### Features * Add SQLite retrieve_online_documents_v2 ([#5032](#5032)) ([0fffe21](0fffe21)) * Adding Click command to display configuration details ([#5036](#5036)) ([ae68e4d](ae68e4d)) * Adding volumes and volumeMounts support to Feature Store CR. ([#4983](#4983)) ([ec6f1b7](ec6f1b7)) * Moving the job to seperate action so that we can test it easily. ([#5013](#5013)) ([b9325b7](b9325b7)) * Operator - make server container creation explicit in the CR ([#5024](#5024)) ([b16fb40](b16fb40))
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
We are using
psycopg3in case we use Postgres for the online store. If you create connections withpsycopg3using a context manager (i.e. awith ...block), it will automatically commit the transaction once you exit the context. However, infeastwe are manually creating (and in the case of connection pools, putting) connections, so we need to ensure that we also explicitly callcommitafter firing queries to the database.The
commitstatements were missing for (async) read methods of thePostgresOnlineStoreand theteardownmethod. For the read statements, this wasn't an issue as we just do aSELECT. However, it does add warning statements and quite a bit of latency as transactions needed to be rolled back.Hence, for those read methods, I introduced an optional
autocommitto the_get_connand_get_conn_asyncmethods that defaults toFalse. We are setting it toTruein our read methods, so that prevents us from having to callcommitand further optimizes the performance. In my local tests,online_read_asyncperformance improved by 20-30% when calling explicitly commit, and 40-70% when enabling autocommit (depending on feature view).Which issue(s) this PR fixes:
Fixes #5038