Skip to content

Commit ba7c7bb

Browse files
committed
docs: document warmup_connections config and increase test coverage
Signed-off-by: Nithin <kumbam.nithingoud@gmail.com>
1 parent e376322 commit ba7c7bb

3 files changed

Lines changed: 12 additions & 0 deletions

File tree

docs/how-to-guides/online-server-performance-tuning.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -324,6 +324,7 @@ online_store:
324324
batch_size: 100
325325
max_read_workers: 10
326326
consistent_reads: false
327+
warmup_connections: true
327328
max_pool_connections: 100
328329
keepalive_timeout: 30.0
329330
connect_timeout: 3
@@ -337,6 +338,7 @@ Key knobs:
337338
- **`batch_size`**: DynamoDB's `BatchGetItem` accepts up to 100 items per request. For 500 entities, this means 5 batches. Keep at 100 unless hitting the 16 MB response limit.
338339
- **`max_read_workers`**: Controls parallelism for batch reads. With 10 workers, those 5 batches run concurrently (~10 ms) instead of sequentially (~50 ms).
339340
- **`consistent_reads: false`**: Eventually consistent reads are faster and cheaper. Use `true` only if you need read-after-write consistency.
341+
- **`warmup_connections: true`**: Pre-warms the DynamoDB connection pool on server startup by making a lightweight call (`describe_limits`). This avoids a cold-start latency penalty (~20ms) on the very first feature request.
340342
- **`max_pool_connections`**: Increase for high-throughput deployments to improve HTTP connection reuse to the DynamoDB endpoint.
341343
- **`keepalive_timeout`**: Longer keep-alive reduces TLS handshake overhead on reused connections.
342344
- **`connect_timeout` / `read_timeout`**: Lower values fail fast, improving p99. Set aggressively if your retry strategy covers transient failures.

docs/reference/online-stores/dynamodb.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ online_store:
3737
batch_size: 100
3838
max_read_workers: 10
3939
consistent_reads: false
40+
warmup_connections: true
4041
```
4142
{% endcode %}
4243
@@ -49,6 +50,7 @@ online_store:
4950
| `batch_size` | int | `100` | Number of items per BatchGetItem/BatchWriteItem request (max 100) |
5051
| `max_read_workers` | int | `10` | Maximum parallel threads for batch read operations. Higher values improve throughput for large batch reads but increase resource usage |
5152
| `consistent_reads` | bool | `false` | Whether to use strongly consistent reads (higher latency, guaranteed latest data) |
53+
| `warmup_connections` | bool | `false` | Whether to pre-warm the async connection pool on startup with a lightweight call (`describe_limits`) |
5254
| `tags` | dict | `null` | AWS resource tags added to each table |
5355
| `session_based_auth` | bool | `false` | Use AWS session-based client authentication |
5456

@@ -63,6 +65,8 @@ For high-throughput workloads with large entity counts, increase `max_read_worke
6365

6466
**Batch Size**: Increase `batch_size` up to 100 to reduce the number of API calls. However, larger batches may hit DynamoDB's 16MB response limit for tables with large feature values.
6567

68+
**Connection Warmup**: The DynamoDB async client does not establish actual TCP/TLS connections to the AWS endpoint on initialization. The very first feature retrieval request is penalized with a cold-start overhead (~20ms). Setting `warmup_connections: true` establishes the TCP connection pool during server startup.
69+
6670
## Permissions
6771

6872
Feast requires the following permissions in order to execute commands for DynamoDB online store:

sdk/python/tests/unit/infra/online_store/test_dynamodb_online_store.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -136,6 +136,12 @@ async def test_dynamodb_online_store_warmup_connections():
136136
await online_store.initialize(config_no_warmup)
137137
mock_client.describe_limits.assert_not_called()
138138

139+
# Test case 3: warmup_connections=True and describe_limits raises an exception (should catch and log warning)
140+
mock_client.describe_limits.reset_mock()
141+
mock_client.describe_limits.side_effect = Exception("Connection failed")
142+
await online_store.initialize(config_warmup)
143+
mock_client.describe_limits.assert_called_once()
144+
139145

140146
def test_dynamodb_online_store_config_dynamodb_client(dynamodb_online_store):
141147
"""Test DynamoDBOnlineStoreConfig configure DynamoDB client with endpoint_url."""

0 commit comments

Comments
 (0)