Summary
The feature server crashes on every /get-online-features and /retrieve-online-documents request
when running with protobuf >= 7.34.0. The float_precision parameter was deprecated in protobuf
6.32.x and removed in 7.34.0. Since Feast's pyproject.toml allows protobuf >= 4.24.0,
fresh installs (including the official quay.io/feastdev/feature-server:latest image) pull
protobuf 7.34.1, which does not accept this argument.
Error
TypeError: MessageToDict() got an unexpected keyword argument 'float_precision'
Full traceback from Gunicorn worker:
File "/opt/app-root/lib64/python3.12/site-packages/feast/feature_server.py", line 397 response_dict = await run_in_threadpool( MessageToDict, response.proto, preserving_proto_field_name=True, float_precision=18, # <-- removed in protobuf 7.34.0 )
The worker process crashes, Gunicorn master respawns it, and the next request crashes again —
making the online feature server completely non-functional.
Affected Code
sdk/python/feast/feature_server.py — two call sites:
get_online_features endpoint (line ~398–401)
retrieve_online_documents endpoint (line ~440–443)
Both pass float_precision=18 to google.protobuf.json_format.MessageToDict.
Environment
- Feast version: 0.63.0
- Image:
quay.io/feastdev/feature-server:latest (also quay.io/feastdev-ci/feature-server:develop)
- Python: 3.12.12
- protobuf: 7.34.1 (installed by pip as allowed by
protobuf>=4.24.0 in pyproject.toml)
Reproducer
# 1. Install feast with latest protobuf
pip install feast
# 2. Verify protobuf version
python -c "import google.protobuf; print(google.protobuf.__version__)"
# Output: 7.34.1
# 3. Reproduce the crash directly
python -c "
from google.protobuf.json_format import MessageToDict
from feast.protos.feast.serving.ServingService_pb2 import GetOnlineFeaturesResponse
resp = GetOnlineFeaturesResponse()
MessageToDict(resp, preserving_proto_field_name=True, float_precision=18)
"
# Output:
# TypeError: MessageToDict() got an unexpected keyword argument 'float_precision'
Or with a running feature server, Any POST to the online server will crash the worker:
curl -X POST https://<feast-server>/get-online-features \
-H "Content-Type: application/json" \
-d '{"features": ["my_fv:my_feature"], "entities": {"entity_id": [1]}}'
# Returns: 500 Internal Server Error
Summary
The feature server crashes on every
/get-online-featuresand/retrieve-online-documentsrequestwhen running with
protobuf >= 7.34.0. Thefloat_precisionparameter was deprecated in protobuf6.32.x and removed in 7.34.0. Since Feast's
pyproject.tomlallowsprotobuf >= 4.24.0,fresh installs (including the official
quay.io/feastdev/feature-server:latestimage) pullprotobuf 7.34.1, which does not accept this argument.
Error
TypeError: MessageToDict() got an unexpected keyword argument 'float_precision'Full traceback from Gunicorn worker:
File "/opt/app-root/lib64/python3.12/site-packages/feast/feature_server.py", line 397 response_dict = await run_in_threadpool( MessageToDict, response.proto, preserving_proto_field_name=True, float_precision=18, # <-- removed in protobuf 7.34.0 )The worker process crashes, Gunicorn master respawns it, and the next request crashes again —
making the online feature server completely non-functional.
Affected Code
sdk/python/feast/feature_server.py— two call sites:get_online_featuresendpoint (line ~398–401)retrieve_online_documentsendpoint (line ~440–443)Both pass
float_precision=18togoogle.protobuf.json_format.MessageToDict.Environment
quay.io/feastdev/feature-server:latest(alsoquay.io/feastdev-ci/feature-server:develop)protobuf>=4.24.0in pyproject.toml)Reproducer
Or with a running feature server, Any POST to the online server will crash the worker: