-
Notifications
You must be signed in to change notification settings - Fork 1.2k
feat: Add per-service scaling & gRPC support #5934
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
feat: Add per-service scaling & gRPC support #5934
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
e340591 to
f7f7845
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
f7f7845 to
206c701
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
206c701 to
38da893
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
38da893 to
c87e4be
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
c87e4be to
9ff2621
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
5ff72f8 to
99b5ca7
Compare
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
|
@Srihari1192 Can we please create an image and test these operator changes ? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
…:jatin5251/feast into feat-feast-operator-mcp-grpc-per-service
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
Signed-off-by: Jatin Kumar <jatink.5251@gmail.com>
| cmd = append(cmd, "--registry_ttl_sec", strconv.Itoa(int(*grpcCfg.RegistryTTLSeconds))) | ||
| } | ||
| } | ||
| if tls := feast.getTlsConfigs(OnlineGrpcFeastType); tls.IsTLS() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
feast listen doesn't support --key/--cert options
| }, | ||
| } | ||
| } | ||
| if feast.onlineGrpcOpenshiftTls() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
gRPC server has NO TLS support at all and here we are trying to auto-enable broken TLS
| } | ||
|
|
||
| func (feast *FeastServices) setContainers(podSpec *corev1.PodSpec) error { | ||
| func (feast *FeastServices) setContainers(podSpec *corev1.PodSpec, feastType FeastServiceType) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see this is breaking change and there is no clear migration path from old deployment to new multiple deployments, single container each.
It keeps the old feast-my-store deployment. After upgrade:
- Old feast-my-store (4 containers) keeps running
- New feast-my-store-* (4-5 deployments) are created
- Both run simultaneously = conflicts, duplicate services, data corruption risks
|
@jatin5251 If possible can we divide the PR into 2 phases, mainly because this contains core architecture changes.
Thoughts? |
| func (feast *FeastServices) mountPvcConfig(podSpec *corev1.PodSpec, pvcConfig *feastdevv1.PvcConfig, feastType FeastServiceType) { | ||
| if podSpec != nil && pvcConfig != nil { | ||
| volName := feast.initPVC(feastType).Name | ||
| volName := feast.initPVC(feast.pvcFeastType(feastType)).Name |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With separate PVCs, I think it will break file-based persistence. Each pod only has access to its own storage. While materialization with file-based (parquet files), Online pod can't access offline PVC.
What this PR does / why we need it:
feature_serverconfig passthrough in the FeatureStore CRD to enable MCP support in the feature server config.Which issue(s) this PR fixes:
(If required i can create Github issue)
Misc