Datahub Action - Hello World - Invalid magic byte #15034
Description
Describe the bug
When testing Datahub Actions following the Hello World tutorial from the documentation, I can't solve the error
ERROR {datahub_actions.plugin.source.kafka.kafka_event_source:176} - Kafka consume error: KafkaError{code=_VALUE_DESERIALIZATION,val=-159,str="Invalid magic byte"}
To Reproduce
Steps to reproduce the behavior:
- with a Dathub installed with
datahub docker quickstart - using the following yaml
# 1. Action Pipeline Name
name: "hello_world"
# 2. Event Source: Where to source event from.
source:
type: "kafka"
config:
connection:
bootstrap: localhost:9092
schema_registry_url: "http://localhost:8080/schema-registry/api/"
topic_routes:
due: DataHubUsageEvent_v1
# 3. Action: What action to take on events.
action:
type: "hello_world"
config: {}also tested with schema_registry_url: "http://localhost:8080"
- run
datahub actions run -c ~/datahub_actions/hello_world_test.yaml
that start well with
[2025-10-17 14:12:45,427] INFO {datahub_actions.cli.actions:106} - DataHub Actions version: 1.3.0
[2025-10-17 14:12:45,530] INFO {datahub_actions.cli.actions:178} - Action Pipeline with name 'hello_world' is now running.click something in the Datahub UI
- See error
[2025-10-17 14:12:45,427] INFO {datahub_actions.cli.actions:106} - DataHub Actions version: 1.3.0
[2025-10-17 14:12:45,530] INFO {datahub_actions.cli.actions:178} - Action Pipeline with name 'hello_world' is now running.
[2025-10-17 14:13:01,329] ERROR {datahub_actions.plugin.source.kafka.kafka_event_source:176} - Kafka consume error: KafkaError{code=_VALUE_DESERIALIZATION,val=-159,str="Invalid magic byte"}
Traceback (most recent call last):
File "/home/thales/python-virtual-environments/datahub/lib/python3.10/site-packages/confluent_kafka/deserializing_consumer.py", line 110, in poll
value = self._value_deserializer(value, ctx)
File "/home/thales/python-virtual-environments/datahub/lib/python3.10/site-packages/confluent_kafka/schema_registry/_sync/avro.py", line 521, in __call__
return self.__deserialize(data, ctx)
File "/home/thales/python-virtual-environments/datahub/lib/python3.10/site-packages/confluent_kafka/schema_registry/_sync/avro.py", line 558, in __deserialize
payload = self._schema_id_deserializer(data, ctx, schema_id)
File "/home/thales/python-virtual-environments/datahub/lib/python3.10/site-packages/confluent_kafka/schema_registry/__init__.py", line 211, in dual_schema_id_deserializer
return schema_id.from_bytes(io.BytesIO(payload)) # Parse from payload, return remainder
File "/home/thales/python-virtual-environments/datahub/lib/python3.10/site-packages/confluent_kafka/schema_registry/common/serde.py", line 345, in from_bytes
raise SerializationError("Invalid magic byte")
confluent_kafka.serialization.SerializationError: Invalid magic byte
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/thales/python-virtual-environments/datahub/lib/python3.10/site-packages/datahub_actions/plugin/source/kafka/kafka_event_source.py", line 174, in events
msg = self.consumer.poll(timeout=2.0)
File "/home/thales/python-virtual-environments/datahub/lib/python3.10/site-packages/confluent_kafka/deserializing_consumer.py", line 112, in poll
raise ValueDeserializationError(exception=se, kafka_message=msg)
confluent_kafka.error.ValueDeserializationError: KafkaError{code=_VALUE_DESERIALIZATION,val=-159,str="Invalid magic byte"}Expected behavior
A properly formatted message describing the UI click
Desktop (please complete the following information):
- OS: Ubuntu 22.04.5
- Version
acryl-datahub, version 1.3.0
DataHub Actions version: 1.3.0
Python version: 3.10.12 (main, Aug 15 2025, 14:32:43) [GCC 11.4.0]