Add support for `webhooks` in `--openapi-scopes` #2481

HNygard · 2025-08-15T12:25:57Z

Note: Change made with Github Copilot. Verified locally on own data in addition to the tests created.

This PR adds support for the webhooks scope in the --openapi-scopes CLI option to handle OpenAPI Documents that have a webhooks root element instead of or in addition to paths and components.

Background

According to the OpenAPI 3.1
specification, OpenAPI documents can contain a webhooks root element that describes incoming webhooks. These webhooks have a similar structure to paths - they contain operations with requestBody that have content schema attributes, making them suitable for model generation.

Changes Made

1. Added `Webhooks` to OpenAPIScope enum

class OpenAPIScope(Enum):
    Schemas = "schemas"
    Paths = "paths"
    Tags = "tags"
    Parameters = "parameters"
    Webhooks = "webhooks"  # NEW

2. Implemented webhooks processing in OpenAPI parser Added logic in the `parse_raw()` method to process webhooks similar to how paths are processed:

if OpenAPIScope.Webhooks in self.open_api_scopes:
webhooks: dict[str, dict[str, Any]] = specification.get("webhooks",
{})
    webhooks_path = [*path_parts, "#/webhooks"]
    for webhook_name, methods_ in webhooks.items():
        # Process each webhook operation
        for operation_name, raw_operation in methods.items():
            if operation_name not in OPERATION_NAMES:
                continue
            self.parse_operation(raw_operation, [*path, operation_name])

3. Added comprehensive test coverage

Created tests/data/openapi/webhooks.yaml with a complete OpenAPI spec containing webhooks using dotted naming convention (pet.new, pet.updated)
Added test functions to validate webhooks-only and combined scope processing
Created expected output files for test validation

Usage Examples

# Generate models from webhooks only
datamodel-codegen --input spec.yaml --openapi-scopes webhooks

# Generate models from schemas and webhooks
datamodel-codegen --input spec.yaml --openapi-scopes schemas webhooks

# Generate models from all scopes including webhooks
datamodel-codegen --input spec.yaml --openapi-scopes schemas paths tags
parameters webhooks

Benefits

✅ Support for OpenAPI 3.1+ webhooks specification
✅ Generate Pydantic models for webhook request bodies
✅ Reuse existing component schemas in webhook definitions
✅ Flexible scope combinations (webhooks can be used alone or with other scopes)
✅ Same robust processing as paths (handles references, nested objects, validation, etc.)
✅ Supports dotted webhook naming convention (e.g., pet.new, user.updated)
✅ Backward compatible - no breaking changes to existing functionality

The implementation reuses the existing operation processing logic since webhooks have the same structure as path operations, ensuring consistency and reliability.

Change made with Github Copilot. Verified locally on own data in addition to the tests created.

Fixes #2059

This PR adds support for the `webhooks` scope in the `--openapi-scopes` CLI option to handle OpenAPI Documents that have a `webhooks` root element instead of or in addition to `paths` and `components`. ## Background According to the [OpenAPI 3.1 specification](https://spec.openapis.org/oas/latest.html#oasWebhooks), OpenAPI documents can contain a `webhooks` root element that describes incoming webhooks. These webhooks have a similar structure to paths - they contain operations with `requestBody` that have content `schema` attributes, making them suitable for model generation. ## Changes Made ### 1. Added `Webhooks` to OpenAPIScope enum ```python class OpenAPIScope(Enum): Schemas = "schemas" Paths = "paths" Tags = "tags" Parameters = "parameters" Webhooks = "webhooks" # NEW ``` ### 2. Implemented webhooks processing in OpenAPI parser Added logic in the `parse_raw()` method to process webhooks similar to how paths are processed: ```python if OpenAPIScope.Webhooks in self.open_api_scopes: webhooks: dict[str, dict[str, Any]] = specification.get("webhooks", {}) webhooks_path = [*path_parts, "#/webhooks"] for webhook_name, methods_ in webhooks.items(): # Process each webhook operation for operation_name, raw_operation in methods.items(): if operation_name not in OPERATION_NAMES: continue self.parse_operation(raw_operation, [*path, operation_name]) ``` ### 3. Added comprehensive test coverage - Created `tests/data/openapi/webhooks.yaml` with a complete OpenAPI spec containing webhooks using dotted naming convention (`pet.new`, `pet.updated`) - Added test functions to validate webhooks-only and combined scope processing - Created expected output files for test validation ## Usage Examples ```bash # Generate models from webhooks only datamodel-codegen --input spec.yaml --openapi-scopes webhooks # Generate models from schemas and webhooks datamodel-codegen --input spec.yaml --openapi-scopes schemas webhooks # Generate models from all scopes including webhooks datamodel-codegen --input spec.yaml --openapi-scopes schemas paths tags parameters webhooks ``` ## Benefits - ✅ Support for OpenAPI 3.1+ webhooks specification - ✅ Generate Pydantic models for webhook request bodies - ✅ Reuse existing component schemas in webhook definitions - ✅ Flexible scope combinations (webhooks can be used alone or with other scopes) - ✅ Same robust processing as paths (handles references, nested objects, validation, etc.) - ✅ Supports dotted webhook naming convention (e.g., `pet.new`, `user.updated`) - ✅ Backward compatible - no breaking changes to existing functionality The implementation reuses the existing operation processing logic since webhooks have the same structure as path operations, ensuring consistency and reliability. Change made with Github Copilot. Verified locally on own data in addition to the tests created. Fixes koxudaxi#2059

for more information, see https://pre-commit.ci

Copilot

Pull Request Overview

This PR adds support for OpenAPI webhooks by introducing a new Webhooks scope to the --openapi-scopes CLI option. This enables parsing of webhook operations defined in the webhooks root element of OpenAPI 3.1+ specifications, allowing generation of Pydantic models for webhook request bodies alongside existing paths and schemas.

Added Webhooks enum value to OpenAPIScope
Implemented webhook parsing logic in OpenAPIParser.parse_raw() method
Added comprehensive test coverage with test data and expected outputs

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
`src/datamodel_code_generator/__init__.py`	Added `Webhooks` enum value to `OpenAPIScope`
`src/datamodel_code_generator/parser/openapi.py`	Implemented webhook processing logic in `parse_raw()` method
`tests/parser/test_openapi.py`	Added test functions for webhook parsing functionality
`tests/data/openapi/webhooks.yaml`	Created test OpenAPI spec with webhook definitions
`tests/data/expected/parser/openapi/openapi_parser_webhooks/output.py`	Added expected output for webhook parsing tests

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-08-15T12:26:25Z

src/datamodel_code_generator/parser/openapi.py

+                    methods = self.get_ref_model(methods_["$ref"]) if "$ref" in methods_ else methods_
+                    relative_webhook_name = webhook_name.removeprefix("/")
+                    if relative_webhook_name:
+                        path = [*webhooks_path, relative_webhook_name]


The webhook name processing logic strips leading '/' but webhook names in the test data use dotted notation (e.g., 'pet.new'). This path stripping logic appears unnecessary for webhook names which typically don't start with '/' like URL paths do.

Suggested change

path = [*webhooks_path, relative_webhook_name]

if webhook_name:

path = [*webhooks_path, webhook_name]

codspeed-hq · 2025-08-15T14:23:21Z

CodSpeed Performance Report

Merging #2481 will not alter performance

_{Comparing HNygard:main (6a982e2) with main (82a9295)}

Summary

✅ 32 untouched

koxudaxi

@HNygard Thank you for creating the PR.
Can you fix the test errors?

HNygardStb · 2025-08-21T08:30:03Z

@HNygard Thank you for creating the PR. Can you fix the test errors?

I'm looking into it 👀 👍

HNygard · 2025-09-15T13:08:57Z

@koxudaxi: Hi again! Do I need to do something to make the CI run? I think the test should be fine, but I'm not sure.

codecov · 2025-09-16T15:33:03Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.68%. Comparing base (82a9295) to head (6a982e2).
⚠️ Report is 11 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2481      +/-   ##
==========================================
+ Coverage   97.67%   97.68%   +0.01%     
==========================================
  Files          67       67              
  Lines        8128     8167      +39     
  Branches      854      859       +5     
==========================================
+ Hits         7939     7978      +39     
  Misses        144      144              
  Partials       45       45

Flag	Coverage Δ
unittests	`97.68% <100.00%> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

for more information, see https://pre-commit.ci

koxudaxi · 2025-09-17T13:50:29Z

tests/parser/test_openapi.py

+def test_openapi_parser_webhooks_only() -> None:
+    """Test parsing OpenAPI spec with only webhooks scope."""
+    parser = OpenAPIParser(
+        data_model_field_type=DataModelFieldBase,
+        source=Path(DATA_PATH / "webhooks.yaml"),
+        openapi_scopes=[OpenAPIScope.Webhooks],
+    )
+    result = parser.parse()
+
+    # With only webhooks scope, should not include the base schemas
+    # but should include request models generated from webhooks
+    assert result is not None


Similar to test_openapi_parser_webhooks, please create an output.py file and set up assertions.

koxudaxi · 2025-09-17T13:51:19Z

tests/parser/test_openapi.py

+    # Parse the spec - this should not fail and should ignore non-operation fields
+    result = parser.parse()
+    assert result is not None
+
+    # Verify that the result contains the Pet schema
+    assert "class Pet(BaseModel):" in result
+    assert "id: Optional[int] = None" in result
+    assert "name: Optional[str] = None" in result


Similarly, create output.py and run assert.

koxudaxi · 2025-09-17T13:52:15Z

tests/parser/test_openapi.py

+    spec_content = """
+openapi: 3.1.0
+info:
+  title: Security Test API
+  version: 1.0.0
+security:
+  - api_key: []
+paths:
+  /test:
+    get:
+      # No security defined - should inherit global security
+      responses:
+        '200':
+          description: Success
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  message:
+                    type: string
+components:
+  securitySchemes:
+    api_key:
+      type: apiKey
+      in: header
+      name: X-API-Key
+"""
+
+    parser = OpenAPIParser(
+        data_model_field_type=DataModelFieldBase,
+        source=spec_content,
+        openapi_scopes=[OpenAPIScope.Schemas, OpenAPIScope.Paths],
+    )
+
+    # This should not fail - the security inheritance code should execute
+    result = parser.parse()
+    assert result is not None


Please create the input YAML file and output.py.

koxudaxi · 2025-09-17T13:52:35Z

tests/parser/test_openapi.py

+    spec_content = """
+openapi: 3.1.0
+info:
+  title: Webhook Security Test API
+  version: 1.0.0
+security:
+  - api_key: []
+webhooks:
+  test.event:
+    info: "This should be ignored as it's not an operation"
+    post:
+      # No security defined - should inherit global security
+      requestBody:
+        content:
+          application/json:
+            schema:
+              type: object
+              properties:
+                data:
+                  type: string
+      responses:
+        '200':
+          description: Success
+components:
+  securitySchemes:
+    api_key:
+      type: apiKey
+      in: header
+      name: X-API-Key
+"""
+
+    parser = OpenAPIParser(
+        data_model_field_type=DataModelFieldBase,
+        source=spec_content,
+        openapi_scopes=[OpenAPIScope.Schemas, OpenAPIScope.Webhooks],
+    )
+
+    # This should not fail - the security inheritance code should execute
+    result = parser.parse()
+    assert result is not None


same above.

Copilot AI review requested due to automatic review settings August 15, 2025 12:25

[pre-commit.ci] auto fixes from pre-commit.com hooks

81e4a57

for more information, see https://pre-commit.ci

Copilot AI reviewed Aug 15, 2025

View reviewed changes

Ignore PLR0912

dd918df

koxudaxi requested changes Aug 20, 2025

View reviewed changes

HNygard added 2 commits September 8, 2025 16:04

Removed PetNewPostRequest and PetUpdatedPostRequest classes.

be405ad

Merge branch 'main' into main

cb40ef3

HNygard and others added 3 commits September 17, 2025 13:28

Add test coverage

7153233

[pre-commit.ci] auto fixes from pre-commit.com hooks

586c160

for more information, see https://pre-commit.ci

Readme

6a982e2

koxudaxi reviewed Sep 17, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Add support for `webhooks` in `--openapi-scopes` #2481

Add support for `webhooks` in `--openapi-scopes` #2481

Uh oh!

HNygard commented Aug 15, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Aug 15, 2025

Uh oh!

codspeed-hq bot commented Aug 15, 2025 •

edited

Loading

Uh oh!

koxudaxi left a comment

Uh oh!

HNygardStb commented Aug 21, 2025

Uh oh!

HNygard commented Sep 15, 2025

Uh oh!

codecov bot commented Sep 16, 2025 •

edited

Loading

Uh oh!

koxudaxi Sep 17, 2025

Uh oh!

koxudaxi Sep 17, 2025

Uh oh!

koxudaxi Sep 17, 2025

Uh oh!

koxudaxi Sep 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	path = [*webhooks_path, relative_webhook_name]
	if webhook_name:
	path = [*webhooks_path, webhook_name]

Uh oh!

Add support for webhooks in --openapi-scopes #2481

Are you sure you want to change the base?

Add support for webhooks in --openapi-scopes #2481

Uh oh!

Conversation

HNygard commented Aug 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Background

Changes Made

1. Added Webhooks to OpenAPIScope enum

2. Implemented webhooks processing in OpenAPI parser Added logic in the parse_raw() method to process webhooks similar to how paths are processed:

3. Added comprehensive test coverage

Usage Examples

Benefits

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Aug 15, 2025

Choose a reason for hiding this comment

Uh oh!

codspeed-hq bot commented Aug 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodSpeed Performance Report

Merging #2481 will not alter performance

Summary

Uh oh!

koxudaxi left a comment

Choose a reason for hiding this comment

Uh oh!

HNygardStb commented Aug 21, 2025

Uh oh!

HNygard commented Sep 15, 2025

Uh oh!

codecov bot commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

koxudaxi Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

koxudaxi Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

koxudaxi Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

koxudaxi Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Add support for `webhooks` in `--openapi-scopes` #2481

Add support for `webhooks` in `--openapi-scopes` #2481

HNygard commented Aug 15, 2025 •

edited

Loading

1. Added `Webhooks` to OpenAPIScope enum

2. Implemented webhooks processing in OpenAPI parser Added logic in the `parse_raw()` method to process webhooks similar to how paths are processed:

codspeed-hq bot commented Aug 15, 2025 •

edited

Loading

codecov bot commented Sep 16, 2025 •

edited

Loading