The HTTP Request Layer is the final stage of the RudderStack Python SDK's data pipeline. It is responsible for serializing event batches, applying compression, handling authentication, and executing the network request to the RudderStack data plane.
The primary entry point for this layer is the post() function defined in rudderstack/analytics/request.py. This module abstracts the complexities of the requests library, providing a specialized interface for the Consumer to transmit data.
| Component | Responsibility |
|---|---|
post() | Main function to execute HTTP POST requests to the /v1/batch endpoint. |
_session | A shared requests.Session object used to persist TCP connections (keep-alive). |
DatetimeSerializer | A custom JSON encoder that handles Python date and datetime objects. |
APIError | A specialized exception for handling non-200 HTTP responses from the data plane. |
_gzip_json() | Helper function that compresses JSON payloads using GZIP. |
Sources: rudderstack/analytics/request.py13-14 rudderstack/analytics/request.py16 rudderstack/analytics/request.py65 rudderstack/analytics/request.py77
The following diagram illustrates how a batch of events is transformed into an HTTP request.
Batch Processing and Transmission Flow
Sources: rudderstack/analytics/consumer.py111-133 rudderstack/analytics/request.py16-44
The SDK uses HTTP Basic Authentication. The write_key is provided as the username, and the password field is left empty rudderstack/analytics/request.py22
Every request includes a User-Agent header in the format analytics-python/<VERSION> to identify the SDK version to the RudderStack servers rudderstack/analytics/request.py27
Standard JSON serialization in Python does not support datetime objects. The DatetimeSerializer extends json.JSONEncoder to convert these objects into ISO 8601 strings rudderstack/analytics/request.py77-83 Before transmission, a sentAt timestamp is injected into the root of the payload to allow the server to calculate clock skew rudderstack/analytics/request.py20
If gzip is enabled (default is True in DefaultConfig), the JSON string is encoded to UTF-8 and compressed using the gzip module rudderstack/analytics/request.py57-63 The header Content-Encoding: gzip is added to notify the server of the compression rudderstack/analytics/request.py30-31
Sources: rudderstack/analytics/request.py16-31 rudderstack/analytics/client.py30
The post() function monitors the HTTP status code of the response. If the status is not 200, it attempts to parse the error message from the response body and raises an APIError rudderstack/analytics/request.py46-55
Entity Mapping: Exception Logic
The Consumer uses the APIError status code to determine if a request should be retried:
APIError exception is treated as retryable.Sources: rudderstack/analytics/request.py65-74 rudderstack/analytics/consumer.py114-122
The module initializes a global _session using requests.sessions.Session() rudderstack/analytics/request.py13 This session is shared across all calls to post(), enabling:
proxies are configured in the Client, they are passed through the post() function to the session's request method rudderstack/analytics/request.py40-44Sources: rudderstack/analytics/request.py13 rudderstack/analytics/request.py43-44