Hi there,
Thanks for your question. As discussed internally:

1. Current Support: Our LiteLLMChat wrapper does not support using stream=True to split a single LLM output into multiple messages over time. The current wrapper returns the full response once generated.

2. Technical Considerations: Implementing streaming would require a significant redesign—specifically around how UDFs return results (for instance, allowing them to yield partial outputs). This isn’t supported by the current engine.

3. Kafka and Output Connectors: While you can stream question and answer pairs to systems like Kafka by adjusting how the webserver handles responses, splitting one answer into multiple parts is not currently implemented.