(Deprecated) feat: Support DynamoDB as online store in Go feature server #5462
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
iamcodingcat
changed the title
iamcodingcat
changed the title
jyejare
reviewed
Jun 24, 2025
Contributor
jyejare
left a comment
jyejare
left a comment
There was a problem hiding this comment.
LGTM, have some questions/comments.
| github.com/mattn/go-sqlite3 v1.14.23 | ||
| github.com/pkg/errors v0.9.1 | ||
| github.com/redis/go-redis/v9 v9.6.1 | ||
| github.com/roberson-io/mmh3 v0.0.0-20190729202758-fdfce3ba6225 |
Contributor
There was a problem hiding this comment.
@franciscojavierarceo @tchughesiv How are we treating these external/personal repos as a safe repositories to use ?
Something like govulncheck Github Action should really help.
iamcodingcat
commented
Jun 24, 2025
Contributor
Author
iamcodingcat
left a comment
iamcodingcat
left a comment
There was a problem hiding this comment.
I just add this only one test code for dynamodb as online store. Should I write more test code?
iamcodingcat
commented
Jun 24, 2025
Contributor
Author
iamcodingcat
left a comment
iamcodingcat
left a comment
There was a problem hiding this comment.
I use this same serializeEntityKey method that is used in redis online store. So I think that this method is common. So, I move this method to onlinestore.go.
jyejare
approved these changes
Jun 24, 2025
Contributor
|
@iamcodingcat Please sign the commit, one of the checks are failing. |
Signed-off-by: iamcodingcat <joyh951021@gmail.com>
Signed-off-by: iamcodingcat <joyh951021@gmail.com>
Signed-off-by: iamcodingcat <joyh951021@gmail.com>
Signed-off-by: iamcodingcat <joyh951021@gmail.com>
Signed-off-by: iamcodingcat <joyh951021@gmail.com>
…ture (feast-dev#5422) * Remote Write to Online Store Signed-off-by: jyejare <jyejare@redhat.com> * Comments Resolved and code shortened Signed-off-by: jyejare <jyejare@redhat.com> --------- Signed-off-by: jyejare <jyejare@redhat.com> Signed-off-by: iamcodingcat <joyh951021@gmail.com>
Signed-off-by: iamcodingcat <joyh951021@gmail.com>
…t-dev#5454) * Make batch_source optional in PushSource (feast-dev#5440) Signed-off-by: snehsuresh <snehsuresh02@gmail.com> * docs: make batch_source optional in PushSource Signed-off-by: snehsuresh <snehsuresh02@gmail.com> * refactor: remove comment Signed-off-by: snehsuresh <snehsuresh02@gmail.com> * fix: ensure batch_source is valid DataSource after null check Signed-off-by: snehsuresh <snehsuresh02@gmail.com> --------- Signed-off-by: snehsuresh <snehsuresh02@gmail.com> Signed-off-by: iamcodingcat <joyh951021@gmail.com>
…ing (feast-dev#5446) Signed-off-by: Srihari <svenkata@redhat.com> Signed-off-by: iamcodingcat <joyh951021@gmail.com>
…functionality Signed-off-by: Srihari <svenkata@redhat.com> Signed-off-by: iamcodingcat <joyh951021@gmail.com>
…east-dev#5423) * feat: Added HybridOnlineStore for multi-backend online store routing - Implements HybridOnlineStore, enabling routing of online feature operations to different backends based on a configurable tag (e.g., tribe, team, or project) on the FeatureView. - Adds support for specifying the routing tag name via the 'routing_tag' field in the online_store config, allowing flexible backend selection. - Supports multi-tenancy and flexible data management by allowing multiple online store backends in a single Feast deployment. - added documentation - fixed linter raised issues Signed-off-by: r0b0fyi <renukaprasannakumar.badugu@walmart.com> * feat: Added HybridOnlineStore for multi-backend online store routing - Implements HybridOnlineStore, enabling routing of online feature operations to different backends based on a configurable tag (e.g., tribe, team, or project) on the FeatureView. - Adds support for specifying the routing tag name via the 'routing_tag' field in the online_store config, allowing flexible backend selection. - Supports multi-tenancy and flexible data management by allowing multiple online store backends in a single Feast deployment. - added documentation - fixed linter raised issues Signed-off-by: r0b0fyi <renukaprasannakumar.badugu@walmart.com> --------- Signed-off-by: r0b0fyi <renukaprasannakumar.badugu@walmart.com> Co-authored-by: r0b0fyi <renukaprasannakumar.badugu@walmart.com> Signed-off-by: iamcodingcat <joyh951021@gmail.com>
iamcodingcat
force-pushed
the
feat/go/online-store/dynamodb
branch
from
3c7ac9a to
e123a5d
Compare
Contributor
Author
|
@jyejare I am twisted at git commit history for signing my commits. 😢 I made duplicate PR sync with latest master branch. |
Contributor
Author
|
I close this PR and delete branch |
iamcodingcat
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
For ultimate low latency feature serving, I am using Golang api server for production environment. Moreover, a few months ago, I developed the feature that integrates s3 registry into Golang api server.(#5352 , #5336)
I am using Golang api server with redis as online store. But redis is usually for cache server and have lower SLA than DynamoDB. So I tried to integrate dynamodb as online store in Golang api server and succeed it. I refer the source code from integrating dynamodb in python sdk.
Of course, I am not writing test code but I locally finish testing about this feature. After reviewing my PR, I will soon add test code for this. please review about this for me! Thanks.
For reference, I carefully see compatibility between python and go of
mmh3hash algorithm used for converting entity key to entity id(In feast python sdk, it use this). I use this Golang lib and I check the complete compatibility in this lib with python sdk. I found this issue while investigating this. In my local environment(64-bit little endian), the test was completed. I will test it on my production linux environment on k8s cluster, asap.Which issue(s) this PR fixes:
Golang api server can support DynamoDB as online store. About this, the issue(#3971) is made before. I will support this.
Misc
@shuchu @jyejare cc. @franciscojavierarceo