Repository navigation
Nested pw.Schema #118
Description
Is your feature request related to a problem? Please describe.
Currently, there is no way to define a nested schema using pw.Schema.
Describe the solution you'd like
Be able to use nested pw.schema:
class NestedColumnSchema(pw.Schema):
a: int
b: float
class InputSchema(pw.Schema):
key: int
nested_column: NestedColumnSchema
Describe alternatives you've considered
Currently, a workaround is to use pw.JSON, but the engine cannot verify the schema inside it.
Additional context
Link to issue #117
Activity
danghoangnhan commentedon Jun 1, 2026
+1 — same need here. My data arrives as nested JSON, both from REST APIs and from hierarchical files (JSON Lines). The
pw.Jsonworkaround works for accessing fields, but as the issue notes, the engine can't validate the structure or give me typed columns without manually projecting every leaf with["a"]["b"].as_int().What I'd want is Pydantic-style nesting — declare a
pw.Schemafield whose type is anotherpw.Schema, and have the engine validate the nested object on ingest (rejecting/flagging rows that don't match), the same way it already validates flat columns. Auto-flattening nested fields into dotted/typed columns would also cover most of my cases. This would remove a lot of boilerplate and catch malformed API/file records at the connector boundary instead of deep in the pipeline.