feat: Use ORJSONResponse for faster JSON serialization in feature server #5930
Conversation
franciscojavierarceo
left a comment
franciscojavierarceo
left a comment
There was a problem hiding this comment.
Need to get tests passing but approving to unblock
@franciscojavierarceo Test failures are due to transient dependency sqlglot, related to ibis-project/ibis#11882. I have handled it already in #5928 , will rebase this PR once it's merged. |
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| float_precision=18, | ||
| ) | ||
| return response_dict | ||
| return ORJSONResponse(content=response_dict) |
There was a problem hiding this comment.
🔴 ORJSONResponse serializes NaN/Infinity as null, changing API behavior
Switching from JSONResponse to ORJSONResponse changes how NaN and Infinity float values are serialized in the API response.
Click to expand
Behavior Change
The standard json module (used by JSONResponse) serializes special float values as:
float('nan')→NaNfloat('inf')→Infinityfloat('-inf')→-Infinity
While orjson (used by ORJSONResponse) serializes them as:
float('nan')→nullfloat('inf')→nullfloat('-inf')→null
Impact
The Feast codebase explicitly uses NaN values for missing data. In sdk/python/feast/proto_json.py:99-101:
# Convert each null as NaN.
message.double_list_val.val.extend(
[item if item is not None else float("nan") for item in value]
)And tests in sdk/python/tests/unit/test_proto_json.py:65 verify NaN is preserved:
assertpy.assert_that(feature_vector_json["values"][5][3]).is_nan()With ORJSONResponse, clients that previously received NaN in responses will now receive null, which has different semantics (missing value vs. not-a-number). This is a breaking change for API consumers that rely on distinguishing between null and NaN values.
Was this helpful? React with 👍 or 👎 to provide feedback.
2 participants
What this PR does / why we need it:
This PR improves the latency of the online feature server by using FastAPI's built-in ORJSONResponse instead of the default JSONResponse. This leverages orjson, a fast JSON library written in Rust, for response serialization.
Changes
When returning a dict from a FastAPI endpoint, FastAPI uses Python's standard json module for serialization. By using ORJSONResponse, we leverage orjson which is written in Rust and provides significantly faster JSON encoding.