Skip to content

Commit e022bf8

Browse files
ntkatholentkathole
authored
feat: Add HTTP connection pooling for remote online store client (#5895)
Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
1 parent 05a4fb5 commit e022bf8

File tree

6 files changed

+662
-43
lines changed

6 files changed

+662
-43
lines changed

‎docs/reference/online-stores/remote.md‎

Lines changed: 123 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
## Description
44

5-
This remote online store will let you interact with remote feature server. At this moment this only supports the read operation. You can use this online store and able retrieve online features `store.get_online_features` from remote feature server.
5+
This remote online store lets you interact with a remote feature server. You can use this online store to retrieve online features via `store.get_online_features()` from a remote feature server.
66

77
## Examples
88

@@ -25,6 +25,128 @@ auth:
2525
2626
`cert` is an optional configuration to the public certificate path when the online server starts in TLS(SSL) mode. This may be needed if the online server is started with a self-signed certificate, typically this file ends with `*.crt`, `*.cer`, or `*.pem`.
2727

28+
## Connection Pooling Configuration
29+
30+
The remote online store uses HTTP connection pooling to improve performance by reusing TCP/TLS connections across multiple requests. This significantly reduces latency by avoiding the overhead of establishing new connections for each request.
31+
32+
### Configuration Options
33+
34+
| Option | Type | Default | Description |
35+
|--------|------|---------|-------------|
36+
| `connection_pool_size` | int | 50 | Maximum number of connections to keep in the pool. Increase for high-concurrency workloads. |
37+
| `connection_idle_timeout` | int | 300 | Maximum time in seconds a session can be idle before being closed. Set to `0` to disable idle timeout. |
38+
| `connection_retries` | int | 3 | Number of retries for failed requests with exponential backoff. |
39+
40+
### Example with Connection Pooling
41+
42+
{% code title="feature_store.yaml" %}
43+
```yaml
44+
project: my-local-project
45+
registry: /remote/data/registry.db
46+
provider: local
47+
online_store:
48+
type: remote
49+
path: http://feast-feature-server:80
50+
51+
# Connection pooling configuration (optional)
52+
connection_pool_size: 50 # Max connections in pool
53+
connection_idle_timeout: 300 # Idle timeout in seconds (0 to disable)
54+
connection_retries: 3 # Retry count with exponential backoff
55+
56+
entity_key_serialization_version: 3
57+
auth:
58+
type: no_auth
59+
```
60+
{% endcode %}
61+
62+
### Use Cases
63+
64+
**High-throughput workloads:**
65+
```yaml
66+
online_store:
67+
type: remote
68+
path: http://feast-server:80
69+
connection_pool_size: 100 # More connections for high concurrency
70+
connection_idle_timeout: 600 # 10 minutes idle timeout
71+
connection_retries: 5 # More retries for resilience
72+
```
73+
74+
**Long-running services:**
75+
```yaml
76+
online_store:
77+
type: remote
78+
path: http://feast-server:80
79+
connection_idle_timeout: 0 # Never auto-close session
80+
```
81+
82+
**Resource-constrained environments:**
83+
```yaml
84+
online_store:
85+
type: remote
86+
path: http://feast-server:80
87+
connection_pool_size: 10 # Fewer connections to reduce memory
88+
connection_idle_timeout: 60 # 1 minute timeout
89+
```
90+
91+
### Performance Benefits
92+
93+
Connection pooling provides significant latency improvements:
94+
95+
- **Without pooling**: Each request requires a new TCP connection (~10-50ms) and TLS handshake (~30-100ms)
96+
- **With pooling**: Subsequent requests reuse existing connections, eliminating connection overhead
97+
98+
This is especially beneficial for:
99+
- High-frequency feature retrieval in real-time inference pipelines
100+
- Batch processing with many sequential `get_online_features()` calls
101+
- Services with authentication enabled (reduces token refresh overhead)
102+
103+
### Session Cleanup
104+
105+
The HTTP session is automatically managed with idle timeout, but you can also explicitly close it when your application is shutting down or when you want to release resources.
106+
107+
#### Using FeatureStore context manager (recommended)
108+
109+
The recommended way to ensure proper cleanup is to use the `FeatureStore` as a context manager:
110+
111+
```python
112+
from feast import FeatureStore
113+
114+
# Session is automatically closed when exiting the context
115+
with FeatureStore(repo_path=".") as store:
116+
features = store.get_online_features(
117+
features=["driver_hourly_stats:conv_rate"],
118+
entity_rows=[{"driver_id": 1001}]
119+
)
120+
```
121+
122+
#### Explicit cleanup
123+
124+
You can also explicitly close the session by calling `close()` on the `FeatureStore`:
125+
126+
```python
127+
from feast import FeatureStore
128+
129+
store = FeatureStore(repo_path=".")
130+
try:
131+
features = store.get_online_features(
132+
features=["driver_hourly_stats:conv_rate"],
133+
entity_rows=[{"driver_id": 1001}]
134+
)
135+
finally:
136+
store.close() # Closes HTTP session and releases resources
137+
```
138+
139+
#### Direct session management
140+
141+
For advanced use cases, you can directly manage the HTTP session via `HttpSessionManager`:
142+
143+
```python
144+
from feast.permissions.client.http_auth_requests_wrapper import HttpSessionManager
145+
146+
# Close the cached HTTP session
147+
HttpSessionManager.close_session()
148+
```
149+
28150
## How to configure Authentication and Authorization
29151
Please refer the [page](./../../../docs/getting-started/concepts/permission.md) for more details on how to configure authentication and authorization.
30152

‎sdk/python/feast/infra/online_stores/remote.py‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323
from feast import Entity, FeatureView, RepoConfig
2424
from feast.infra.online_stores.helpers import _to_naive_utc
2525
from feast.infra.online_stores.online_store import OnlineStore
26+
from feast.permissions.client.http_auth_requests_wrapper import HttpSessionManager
2627
from feast.protos.feast.types.EntityKey_pb2 import EntityKey as EntityKeyProto
2728
from feast.protos.feast.types.Value_pb2 import Value as ValueProto
2829
from feast.repo_config import FeastConfigBaseModel
@@ -51,6 +52,18 @@ class RemoteOnlineStoreConfig(FeastConfigBaseModel):
5152
""" str: Path to the public certificate when the online server starts in TLS(SSL) mode. This may be needed if the online server started with a self-signed certificate, typically this file ends with `*.crt`, `*.cer`, or `*.pem`.
5253
If type is 'remote', then this configuration is needed to connect to remote online server in TLS mode. """
5354

55+
# Connection pooling configuration
56+
connection_pool_size: int = 50
57+
""" int: Maximum number of connections to keep in the pool (default 50).
58+
Increase for high-concurrency workloads. """
59+
60+
connection_idle_timeout: int = 300
61+
""" int: Maximum time in seconds a session can be idle before being closed (default 300 = 5 minutes).
62+
Set to 0 to disable idle timeout. """
63+
64+
connection_retries: int = 3
65+
""" int: Number of retries for failed requests with exponential backoff (default 3). """
66+
5467

5568
class RemoteOnlineStore(OnlineStore):
5669
"""
@@ -544,6 +557,20 @@ def teardown(
544557
):
545558
pass
546559

560+
async def close(self) -> None:
561+
"""
562+
Close the HTTP session and release connection pool resources.
563+
564+
This method is called automatically when FeatureStore.close() is invoked.
565+
It cleans up the cached HTTP session used for connection pooling.
566+
567+
Note: Since the session is shared globally, calling close() will affect
568+
all RemoteOnlineStore instances in the same process. This is typically
569+
fine for SDK usage where there's usually one FeatureStore per process.
570+
"""
571+
HttpSessionManager.close_session()
572+
logger.debug("RemoteOnlineStore HTTP session closed")
573+
547574

548575
@rest_error_handling_decorator
549576
def get_remote_online_features(

0 commit comments

Comments
 (0)