-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Description
Is your feature request related to a problem? Please describe.
I'm trying to enable metrics in my Feast deployment using the Feast Operator, but there's no clear way to pass the --metrics flag to the online container. The documentation provides examples for Helm chart deployment but not for Operator-based deployments. When attempting to use a custom image with an entrypoint that includes the --metrics flag, I found that the Operator doesn't respect this entrypoint configuration.
Describe the solution you'd like
Add a configuration option in the Feast Operator specification to enable metrics for the online server. Ideally, this would be a simple boolean flag in the CustomResource definition that, when set to true, would add the --metrics flag to the command arguments of the online container.
Describe alternatives you've considered
I've tried creating a custom Docker image with an ENTRYPOINT that includes the --metrics flag, but the Operator seems to override this with its own command. Here's my Dockerfile:
FROM quay.io/feastdev/feature-server:0.49.0
# Install ddtrace with OpenTelemetry support
RUN pip install 'ddtrace[opentelemetry]>=1.0.0'
# Add environmental configurations
ENV DD_TRACE_ENABLED=true
ENV DD_LOGS_INJECTION=true
ENV DD_RUNTIME_METRICS_ENABLED=true
# Use feast command directly as the entrypoint with metrics flag
ENTRYPOINT ["feast", "serve", "-h", "0.0.0.0", "-p", "6566", "--metrics"]
Additional context
When examining the pods created by the Operator, I can see that my custom entrypoint is being ignored. The Operator is setting its own command:
Containers:
online:
Container ID: containerd://f6c09abfd39d85a60bba9bc5290eca07a0c8b5120ab2e6fc16ee2676cd59d18f
Image: us-central1-docker.pkg.dev/artifacts/vertex-services-docker/feast-staging:sha-41c10
Image ID: us-central1-docker.pkg.dev/artifacts/vertex-services-docker/feast-staging@sha256:0956b2af3fc7bc6cca143f3a886466989d8bb0827fb2a0f1e60d26990fe4d744
Port: 6566/TCP
Host Port: 0/TCP
Command:
feast
--log-level
DEBUG
serve
-h
0.0.0.0
-p
6566
The Helm chart deployment supports metrics through --set metric=true, but there doesn't seem to be an equivalent option for the Operator. Clear documentation on how to enable metrics with the Feast Operator would also be helpful.