feat: multi-pass type name resolution#2213
Conversation
|
Hello! The TMF spec defines The requestBody variant has multiple content types:
These are all valid, standardized media types. The problem is in contentTypeSuffix() it uses strings.Contains(ct, "json") case strings.Contains(ct, "json"):
return "JSON"This causes a loop in resolveCollisions. I think using exact matches for the common media types instead of substring matching, func contentTypeSuffix(ct string) string {
if ct == "" {
return ""
}
ct = strings.ToLower(ct)
switch ct {
case "application/json":
return "JSON"
case "application/xml", "text/xml":
return "XML"
case "application/x-www-form-urlencoded", "multipart/form-data":
return "Form"
case "text/plain":
return "Text"
case "application/octet-stream":
return "Binary"
case "application/yaml", "application/x-yaml":
return "YAML"
default:
return mediaTypeToCamelCase(ct)
}
}Additionally, I think strategyContextSuffix should track whether a context suffix has already been applied to prevent the loop. What do you think about that? Thank you for your work! |
|
Thank you for finding this. i will add it to my test cases. I'm trying my best to make these changes without breaking backward compatibility, and if I CamelCase-convert the full media type, it's going to change the code generation output for existing users who might not expect their JSON suffix to change to MergePatchJSON or something. I'm back-porting these fixes from an experiment that I am doing, and I think the only thing I can really do is delegate the choice of converting the full media type to a short name to the user via configuration options. Look here at the documentation for Whenever I make what I think are simple test cases that are representative, people in the real world have more complex usage :) I never really thought about returning multiple JSON models from a single API. |
|
Ah okay, the Trying to fix that issue from above highlighted more issues I briefly described in my PR on your fork here: mromaszewicz#1 |
|
Thanks for doing that. This code, as is, should resolve your conflict by appending numerical suffixes to the end after all the collisions. I think I'm doing the multi-pass wrong. Each strategy needs to be applied globally over all schemas before trying the next one. |
When multiple content types map to the same short suffix (e.g., application/json, application/merge-patch+json, and application/json-patch+json all mapping to "JSON"), the per-group strategy cascade caused an infinite oscillation: context suffix appended "RequestBody", then content type suffix appended "JSON", then context suffix fired again because the name no longer ended in "RequestBody", ad infinitum. Fix by restructuring resolveCollisions to apply each strategy as a global phase: exhaust one strategy across ALL colliding groups (re-checking for new collisions after each pass) before advancing to the next strategy. This ensures numeric fallback is reached when earlier strategies cannot disambiguate. Also fix resolvedNameForComponent to accept an optional content type for exact matching, so each media type variant of a requestBody or response gets its own resolved name instead of all variants receiving whichever name the map iterator returns first. Adds Pattern H test case (TMF622 scenario from PR oapi-codegen#2213): a component that exists in both schemas and requestBodies where the requestBody has 3 content types that all collapse to the "JSON" short name. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@mitsaucepls , please try this latest patch on your schema. I've added your specific example as a test case, and it now works. When, after all rounds of conflict resolution fail, the numeric fallback catches this, and you end up with The problem was that you can't try conflict resolution strategies on each conflict group, you must run each strategy globally until it makes no more changes, before moving onto the next one. When you rename things in one conflict group, you may actually rename them into conflict with another schema. This problem is very, very complicated, which is why I avoided it until now. Thanks for those PR's you are sending, but they're not the right fix for the issue because they can put the new names into conflict with other stuff. This PR doesn't contain the short names map, that will be a separate pull request which will let you avoid the numerical suffixes and replace them with more meaningful content type abbreviations. |
7773945 to
fb9c473
Compare
When multiple content types map to the same short suffix (e.g., application/json, application/merge-patch+json, and application/json-patch+json all mapping to "JSON"), the per-group strategy cascade caused an infinite oscillation: context suffix appended "RequestBody", then content type suffix appended "JSON", then context suffix fired again because the name no longer ended in "RequestBody", ad infinitum. Fix by restructuring resolveCollisions to apply each strategy as a global phase: exhaust one strategy across ALL colliding groups (re-checking for new collisions after each pass) before advancing to the next strategy. This ensures numeric fallback is reached when earlier strategies cannot disambiguate. Also fix resolvedNameForComponent to accept an optional content type for exact matching, so each media type variant of a requestBody or response gets its own resolved name instead of all variants receiving whichever name the map iterator returns first. Adds Pattern H test case (TMF622 scenario from PR oapi-codegen#2213): a component that exists in both schemas and requestBodies where the requestBody has 3 content types that all collapse to the "JSON" short name. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fb9c473 to
cad170e
Compare
|
Hey, it definitely generates now! But there are still errors because of undefined types. I found that helper types are not generated and that causes the undefined type behavior. diff --git a/pkg/codegen/codegen.go b/pkg/codegen/codegen.go
index 279ed9d..437af32 100644
--- a/pkg/codegen/codegen.go
+++ b/pkg/codegen/codegen.go
@@ -695,6 +695,7 @@ func GenerateTypesForResponses(t *template.Template, responses openapi3.Response
}
types = append(types, typeDef)
+ types = append(types, goType.AdditionalTypes...)
}
}
return types, nilWithout knowing I can imagine other GenerateTypesFor... functions are also affected, just not in my specific case. |
|
Thanks @mitsaucepls . You found another old issue. Neither Responses or RequestBodies handle the additional types, so they miss generating some models. |
|
Please try it again. If that spec continues to have problems, could you attach it here? I couldn't find the OpenAPI file to download. |
|
I just realized I was on main for my last testing... sorry :D This is the OpenAPI Yaml file Ive been using. TMF622-ProductOrdering-v5.0.0.oas.yaml alternatively downloadable here but it requires an account. This is the config.yaml I used so far: package: api
generate:
echo-server: true
strict-server: true
models: true
output-options:
resolve-type-name-collisions: true
output: gen.api.goWhen using the fix/name-collisions branch there are still some errors. But fewer than before! |
When multiple content types map to the same short suffix (e.g., application/json, application/merge-patch+json, and application/json-patch+json all mapping to "JSON"), the per-group strategy cascade caused an infinite oscillation: context suffix appended "RequestBody", then content type suffix appended "JSON", then context suffix fired again because the name no longer ended in "RequestBody", ad infinitum. Fix by restructuring resolveCollisions to apply each strategy as a global phase: exhaust one strategy across ALL colliding groups (re-checking for new collisions after each pass) before advancing to the next strategy. This ensures numeric fallback is reached when earlier strategies cannot disambiguate. Also fix resolvedNameForComponent to accept an optional content type for exact matching, so each media type variant of a requestBody or response gets its own resolved name instead of all variants receiving whichever name the map iterator returns first. Adds Pattern H test case (TMF622 scenario from PR oapi-codegen#2213): a component that exists in both schemas and requestBodies where the requestBody has 3 content types that all collapse to the "JSON" short name. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
0d74374 to
8107da4
Compare
|
I'm going to split that AdditionalTypes fix out of this PR and into a new one, with a new issue, because they are independent problems. |
8107da4 to
98dfe03
Compare
|
Man, that spec you attached is a doozy. It exposed a few more issues. Now, it generates cleanly with the latest update. |
9ae0501 to
50fd1a4
Compare
50fd1a4 to
dd19a4e
Compare
When multiple content types map to the same short suffix (e.g., application/json, application/merge-patch+json, and application/json-patch+json all mapping to "JSON"), the per-group strategy cascade caused an infinite oscillation: context suffix appended "RequestBody", then content type suffix appended "JSON", then context suffix fired again because the name no longer ended in "RequestBody", ad infinitum. Fix by restructuring resolveCollisions to apply each strategy as a global phase: exhaust one strategy across ALL colliding groups (re-checking for new collisions after each pass) before advancing to the next strategy. This ensures numeric fallback is reached when earlier strategies cannot disambiguate. Also fix resolvedNameForComponent to accept an optional content type for exact matching, so each media type variant of a requestBody or response gets its own resolved name instead of all variants receiving whichever name the map iterator returns first. Adds Pattern H test case (TMF622 scenario from PR oapi-codegen#2213): a component that exists in both schemas and requestBodies where the requestBody has 3 content types that all collapse to the "JSON" short name. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Greptile SummaryThis PR implements a multi-pass type name resolution system to handle collisions across OpenAPI component sections and client response wrappers. The implementation splits code generation into two phases: gathering all schemas with location metadata, then globally resolving naming conflicts using iterative strategies (context suffix, per-schema disambiguation, numeric fallback). Component schemas are privileged and keep bare names when colliding with other types. Key improvements:
The feature is gated behind the existing Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| pkg/codegen/gather.go | New file implementing schema gathering phase - well-structured with clear contexts and comprehensive coverage of all OpenAPI sections |
| pkg/codegen/resolve_names.go | New file implementing multi-pass collision resolution with iterative strategies - solid algorithm preventing oscillation |
| pkg/codegen/codegen.go | Integrates multi-pass resolution, adds helper functions for name lookups, includes content-type path handling for multi-JSON responses |
| pkg/codegen/utils.go | Removes old collision resolution functions (FixDuplicateTypeNames), adds resolved name lookup in refPathToGoTypeSelf |
| pkg/codegen/operations.go | Adds content-type handling in schema paths for multi-JSON responses, uses resolvedNameForRefPath for proper type resolution |
| internal/test/name_conflict_resolution/spec.yaml | Comprehensive integration test spec covering all documented collision patterns from 13+ issues and PRs |
Last reviewed commit: dd19a4e
dd19a4e to
5b965aa
Compare
When multiple content types map to the same short suffix (e.g., application/json, application/merge-patch+json, and application/json-patch+json all mapping to "JSON"), the per-group strategy cascade caused an infinite oscillation: context suffix appended "RequestBody", then content type suffix appended "JSON", then context suffix fired again because the name no longer ended in "RequestBody", ad infinitum. Fix by restructuring resolveCollisions to apply each strategy as a global phase: exhaust one strategy across ALL colliding groups (re-checking for new collisions after each pass) before advancing to the next strategy. This ensures numeric fallback is reached when earlier strategies cannot disambiguate. Also fix resolvedNameForComponent to accept an optional content type for exact matching, so each media type variant of a requestBody or response gets its own resolved name instead of all variants receiving whichever name the map iterator returns first. Adds Pattern H test case (TMF622 scenario from PR oapi-codegen#2213): a component that exists in both schemas and requestBodies where the requestBody has 3 content types that all collapse to the "JSON" short name. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
5b965aa to
5ea3f12
Compare
jamietanna
left a comment
There was a problem hiding this comment.
As a way to review this further, I've taken a look at what happens when we turn this on in a couple of cases:
- against rootly-go, a library I maintain
- by turning it on by default for all users of
oapi-codegen(so we can see example code / tests update)
Unfortunately I'm not sure it's quite ready to go - so I'm going to say let's get this in as part of the next release, not today's.
We're likely going to ship another minor release pretty soon, so this would be a good candidate to get confidence in and then release as part of that
The existing resolve-type-name-collisions changes on main are OK.
rootly-go
We appear to be generating invalid Go code, where the ListIncident response seems to have a space in the name:
❗128732: ListIncidentTypesWithResponse(ctx context.Context, params *ListIncidentTypesParams, reqEditors... RequestEditorFn) (*ListIncident TypesResponse, error)
....
136546: type ListIncident TypesResponse struct {
136547: Body []byte
136548: HTTPResponse *http.Response
136549: ApplicationVndAPIJSON200 *IncidentTypeList
136550: }
136551:
136552: // Status returns HTTPResponse.Status
136553: func (r ListIncident TypesResponse) Status() string {
136554: if r.HTTPResponse != nil {
136555: return r.HTTPResponse.Status
136556: }
136557: return http.StatusText(0)
136558: }
136559:
136560: // StatusCode returns HTTPResponse.StatusCode
136561: func (r ListIncident TypesResponse) StatusCode() int {
136562: if r.HTTPResponse != nil {
136563: return r.HTTPResponse.StatusCode
136564: }
136565: return 0
136566: }oapi-codegen's examples
I've pushed this commit which shows the full range of changes
It seems like it may "just" be that:
- the
NameNormalizerisn't being used / initialisms don't seem to be used when configured x-go-namemight not be working?
I've been meaning to use this approach for a long time, because the attempts at avoiding type collisions via structure suffixes or prefixes work sporadically, at best. Conflict resolution is fundamentally a global problem, not a local problem when doing recursive traversal, so this PR splits the code generation into two parts. First, the OAPI document structure is traversed, and all the schemas that we generate are gathered up into a list of candidates, then we do global conflict resolution across the space of all schemas. This allows us to preserve the functionality of things affected by schema name - `$ref`, `required` properties, and so forth. This fixes issue oapi-codegen#1474 (client response wrapper type colliding with a component schema of the same name and improves issue oapi-codegen#200 handling (same name across schemas, parameters, responses, requestBodies, headers). The new system is gated behind the existing `resolve-type-name-collisions` output option. When disabled, behavior is unchanged, oapi-codegen exits with an error. This flag is default false, so there is no behavior change to oapi-codegen unless it's specified. All current test files regenerate without any differences. Added a comprehensive test which reproduces the scenarios in all the PR's and Issues below, and adds a few more, to make sure that references to renamed targets are also correct.. Key changes: - gather.go: walks entire spec collecting schemas with location metadata - resolve_names.go: assigns unique names via context suffix, per-schema disambiguation, and numeric fallback strategies - Component schemas are privileged and keep bare names on collision - Client response wrapper types now participate in collision detection - Removed ComponentType/DefinedComp from Schema struct - Removed FixDuplicateTypeNames and related functions from utils.go Obsoletes issues: - oapi-codegen#1474 Schema name vs client wrapper (CreateChatCompletionResponse) - oapi-codegen#1713 Schema name vs client wrapper (CreateBlueprintResponse) - oapi-codegen#1450 Schema name vs client wrapper (DeleteBusinessResponse) - oapi-codegen#2097 Path response type vs schema definition (Status) - oapi-codegen#255 Endpoint path vs response type (QueryResponse) - oapi-codegen#899 Duplicate types from response wrapper vs schema (AccessListResponse) - oapi-codegen#1357 Schema vs operationId response (ListAssistantsResponse, OpenAI spec) - oapi-codegen#254 Cross-section: requestBodies vs schemas (Pet) - oapi-codegen#407 Cross-section: requestBodies vs schemas (myThing) - oapi-codegen#1881 Cross-section: requestBodies with multiple content types Obsoletes PRs: - oapi-codegen#292 Parameter structures params postfix (superseded by context suffix) - oapi-codegen#1005 Fix generate equals structs (superseded by multi-pass resolution) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> EOF )
When multiple content types map to the same short suffix (e.g., application/json, application/merge-patch+json, and application/json-patch+json all mapping to "JSON"), the per-group strategy cascade caused an infinite oscillation: context suffix appended "RequestBody", then content type suffix appended "JSON", then context suffix fired again because the name no longer ended in "RequestBody", ad infinitum. Fix by restructuring resolveCollisions to apply each strategy as a global phase: exhaust one strategy across ALL colliding groups (re-checking for new collisions after each pass) before advancing to the next strategy. This ensures numeric fallback is reached when earlier strategies cannot disambiguate. Also fix resolvedNameForComponent to accept an optional content type for exact matching, so each media type variant of a requestBody or response gets its own resolved name instead of all variants receiving whichever name the map iterator returns first. Adds Pattern H test case (TMF622 scenario from PR oapi-codegen#2213): a component that exists in both schemas and requestBodies where the requestBody has 3 content types that all collapse to the "JSON" short name. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add test case from oapi-codegen-exp#14: an inline response object whose properties $ref component schemas with x-go-type: string. In the experimental rewrite (libopenapi), this caused duplicate type declarations because libopenapi copies extensions from $ref targets. V2 (kin-openapi) handles this correctly, but the test guards against future regressions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a component response has multiple JSON content types (e.g., application/json, application/json-patch+json, application/merge-patch+json), three bugs produced uncompilable code: 1. Duplicate inline type declarations: GenerateGoSchema was called with a path of [operationId, responseName] (or [responseName] in the component phase), which doesn't include the content type. When multiple content types share oneOf schemas with inline elements, they produce identically-named AdditionalTypes. If the schemas differ, the result is conflicting declarations; if they're the same, duplicate declarations. Fix: include a mediaTypeToCamelCase(contentType) segment in the schema path when jsonCount > 1, giving each content type's inline types unique names. This is guarded by jsonCount > 1 for backward compatibility. 2. Undefined types in unmarshal code: GetResponseTypeDefinitions used RefPathToGoType to resolve $ref paths like #/components/responses/200Resource_Patch, but without content type context. resolvedNameForComponent fell into a non-deterministic prefix match, returning an arbitrary per-content-type base name that didn't match any defined type when mediaTypeToCamelCase was appended. Fix: add resolvedNameForRefPath helper that parses the $ref path and delegates to resolvedNameForComponent with the specific content type for exact matching. 3. Mismatched types in response struct fields: same root cause as bug 2 — the struct field types were derived from the same non-deterministic resolution path. Additionally, promote AdditionalTypes from GenerateTypesForResponses and GenerateTypesForRequestBodies to the top-level type list, matching the existing pattern in GenerateTypesForSchemas. Without this, inline types (e.g., oneOf union members, nested objects with additionalProperties) defined inside response/requestBody schemas were silently dropped from the output. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Yeah, you're right. This code path ignores the name overrides and the name normalizer function. It's an oversight on my part. I'm going to change it so that user provided names are "pinned", even if that causes conflict, we don't want to second-guess them, but other names will change around them. Fixing the initialisms is easy. Another update, with test coverage for all these scenarios, is coming soon. |
The collision resolver had two bugs when resolve-type-name-collisions was enabled: 1. x-go-name was ignored: generateCandidateName() never consulted the x-go-name extension, so schemas/responses/requestBodies/parameters/ headers with explicit Go name overrides would lose them during collision resolution. 2. Client wrapper names bypassed the name normalizer: generateCandidateName() used UppercaseFirstCharacter(operationID) instead of SchemaNameToTypeName(operationID), so configured normalizers (e.g. ToCamelCaseWithInitialisms) were not applied to client response wrapper type names. Changes: - Add GoNameOverride field to GatheredSchema, populated from x-go-name on the parent container (schema, response, requestBody, parameter, header) when the component is not a $ref - Add extractGoNameOverride() helper to read x-go-name from extensions - Add Pinned field to ResolvedName; pinned names are returned as-is from generateCandidateName() and skipped by all collision resolution strategies (context suffix, content type, status code, param index, numeric fallback) - Fix client wrapper candidate name to use SchemaNameToTypeName() - Add unit tests for GoNameOverride population and pinning behaviour - Add integration test patterns K/L/M (x-go-name on schema, response, and requestBody) and regenerate Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
76c22fe to
a46bd1a
Compare
|
I've regenerated what this looks like as on-by-default and the changes look good, as well as with rootly-go, thanks! |
I've been meaning to use this approach for a long time, because the attempts at avoiding type collisions via structure suffixes or prefixes work sporadically, at best.
Conflict resolution is fundamentally a global problem, not a local problem when doing recursive traversal, so this PR splits the code generation into two parts. First, the OAPI document structure is traversed, and all the schemas that we generate are gathered up into a list of candidates, then we do global conflict resolution across the space of all schemas. This allows us to preserve the functionality of things affected by schema name -
$ref,requiredproperties, and so forth.This fixes issue #1474 (client response wrapper type colliding with a component schema of the same name and improves issue #200 handling (same name across schemas, parameters, responses, requestBodies, headers).
The new system is gated behind the existing
resolve-type-name-collisionsoutput option. When disabled, behavior is unchanged, oapi-codegen exits with an error. This flag is default false, so there is no behavior change to oapi-codegen unless it's specified. All current test files regenerate without any differences.Added a comprehensive test which reproduces the scenarios in all the PR's and Issues below, and adds a few more, to make sure that references to renamed targets are also correct..
Key changes:
Obsoletes issues:
Obsoletes PRs:
EOF
)