You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
References to local files or external files outside of the file being parsed fail to parse correctly. The failure results in the properties being parsed to just be completely ignored.
$ref: path.to.my.yaml
Fails with diagnostic error: The reference string 'path.to.my.yaml' has invalid format.
This appears to be caused by OpenApiV3VersionService.ConvertToOpenApiReference(..) expecting ReferenceType? == null for external references. However, OpenApiV3Deserializer is blindly passing in ReferenceType.Schema.
To clarify the use case here. It doesn't matter to me whether OpenAPI.NET can resolve the reference or not, I'm just trying to use it to parse a set of OpenAPI documents. This issue causes me a problem because it throws away the properties & the reference... which should be able to at least be parsed into the OpenAPI C# objects. It seems like this issue would also block any progress on the 'Workspace' feature or resolving of external references.
var reader = new OpenApiStreamReader(new OpenApiReaderSettings() {
ReferenceResolution = ReferenceResolutionSetting.DoNotResolveReferences
});
var modelSchema = reader.ReadFragment<OpenApiSchema>(stream, OpenApiSpecVersion.OpenApi3_0, out var diagnostic);
modelSchema.Properties is empty.
I would expect modelSchema.Properties["something"].Reference.ExternalResource == "path.to.my.yaml"
Mitchell Cookson (@Cooksauce) I would expect the example you show above to work as you expect. It should also work as you expect if you online resolve internal references. I'll take a look and see if I can repro your issue.
The external references should resolve completely if you provide an absolute path. I need to work on resolving relative paths.
I have found root of the problem.
Responsibility to resolve external resource is delegating to IStreamLoader interface.
In the default implementation (DefaultStreamLoader) used Uri for resolve references.
In this case format of correct reference is: file://path.to.my.yaml, but it's wrong for OpenAPI.
And it's not that I expect too.
I have written replacement for DefaultStreamLoader:
publicclassResourceLoaderFixed:IStreamLoader{privatereadonlystring_mainFilePath;privatereadonlyHttpClient_httpClient=newHttpClient();publicResourceLoaderFixed(stringmainFilePath){_mainFilePath=mainFilePath;}publicStreamLoad(Uriuri){returnLoadAsync(uri).GetAwaiter().GetResult();}publicasyncTask<Stream>LoadAsync(Uriuri){try{varuriScheme=uri.Scheme;switch(uriScheme){case"file":returnFile.OpenRead(uri.AbsolutePath);case"http":case"https":returnawait_httpClient.GetStreamAsync(uri);default:thrownewArgumentException("Unsupported scheme");}}catch(InvalidOperationException)// it seems that it's path to local file without 'file://' prefix{stringoriginalAbsolutePath=uri.OriginalString;stringcompositeAbsolutePath=Path.Combine(_mainFilePath,uri.OriginalString);stringfinalFilePath;if(File.Exists(originalAbsolutePath)){finalFilePath=originalAbsolutePath;}elseif(File.Exists(compositeAbsolutePath)){finalFilePath=compositeAbsolutePath;}else{thrownewException($"Can't to resolve reference to {originalAbsolutePath}");}returnFile.OpenRead(finalFilePath);}}}
And use it as follows:
varspecificationPath=@"C:\Git_Root\Robocop\Veeam.AC.Web.REST\Specification\models\rest_v3.yaml";varspecificationFolder=Path.GetDirectoryName(specificationPath);// Create a reader that will resolve all referencesvarreader=newOpenApiStreamReader(newOpenApiReaderSettings(){ReferenceResolution=ReferenceResolutionSetting.ResolveAllReferences,CustomExternalLoader=newResourceLoaderFixed(specificationFolder)});ReadResultresult;using(varstream=File.OpenRead(specificationPath)){result=awaitreader.ReadAsync(stream);}
I expected that Micosoft.OpenAPI will resolve all JSON references and inline all files into the one document.
But now the feature works wired.
I found the great tutorial by specification splitting and I can do nothing of that.
In the current implementation you can reference only structural elements of OpenAPI. For example: schema, response, etc.
And furthermore, you need to add a header to each of secondary files:
openapi: 3.0.0info:
title: Components for the main appversion: 1.0.0paths: {}components:
And another bug
Open the primary specification with a reference to a schema in another file.
Save the primary specification.
The reference will be replaced to the reference to the primary file, but the schema doesn't copy to primary file.
Thanks for the additional info, this will be very helpful.
Supporting of referencing external fragments will be supported. However, you can only use $ref to replace referenceable components, not just any arbitrary chunk of JSON. If you want to reference arbitrary chunks of JSON you will need to run the file through a pre-processor.
Inlining is not supposed to happen automatically. There is a setting in the writer if you want to inline on write.
But ReferenceInlineSetting there are only three values: DoNotInlineReferences, InlineLocalReferences and InlineAllReferences.
In my case I want to inline only remote references. 😒
Well, to be quite honest, I just want to split a huge specification (~34k lines in yaml) to smaller parts.
And merge the parts together on preprocessing without losing any information (names of schemes for example).
The bug that I mentioned above can be even helpful.
If I copy all parts from secondary OpenApiDocuments to primary, references will be fix automatically. 😆
Mitchell Cookson (@Cooksauce) I would expect the example you show above to work as you expect. It should also work as you expect if you online resolve internal references. I'll take a look and see if I can repro your issue.
Darrel (@darrelmiller) Thanks for taking a look. To be clear, I'm not concerned even whether or not it can resolve the reference, I just need it to be able to read the $ref string without throwing an error which prevents other properties from being parsed.
Additionally, it's worth noting that $ref: path.to.my.yaml#/ does work, while $ref: path.to.my.yaml does not. IMO, the #/ part should not be required in order for parsing to work (since they mean the same thing essentially). You can see why that happens here. I have have the reader settings set to ReferenceResolutionSetting.DoNotResolveReferences. I've tried both with the latest stable nuget release and the latest preview.
Interestingly, if I use realistic path segments (i.e:)
new OpenApiV3VersionService() .ConvertToOpenApiReference( "externalSchema.json#/components/schemas/Pet", null) .ReferenceV3
...I get "common.yaml#/Pet"
But serializing that reference ends up with "$ref: common.yaml#/Pet" (via new OpenApiV3VersionService() .ConvertToOpenApiReference( "common.yaml#/components/schemas/Pet", null) .SerializeAsYaml(OpenApiSpecVersion.OpenApi3_0)
That is, processing "externalSchema.json#/components/schemas/Pet" when serialized back to a file becomes "$ref: common.yaml#/Pet"
Activity
Cooksauce commentedon May 27, 2021
To clarify the use case here. It doesn't matter to me whether
OpenAPI.NETcan resolve the reference or not, I'm just trying to use it to parse a set of OpenAPI documents. This issue causes me a problem because it throws away the properties & the reference... which should be able to at least be parsed into the OpenAPI C# objects. It seems like this issue would also block any progress on the 'Workspace' feature or resolving of external references.For example:
If you parse this with:
modelSchema.Propertiesis empty.I would expect
modelSchema.Properties["something"].Reference.ExternalResource == "path.to.my.yaml"darrelmiller commentedon May 30, 2021
Mitchell Cookson (@Cooksauce) I would expect the example you show above to work as you expect. It should also work as you expect if you online resolve internal references. I'll take a look and see if I can repro your issue.
The external references should resolve completely if you provide an absolute path. I need to work on resolving relative paths.
V0v1kkk commentedon May 30, 2021
Hello Mitchell Cookson (@Cooksauce) , Darrel (@darrelmiller).
I have been looked at new Workspaces feature too.
I have found root of the problem.
Responsibility to resolve external resource is delegating to
IStreamLoaderinterface.In the default implementation (
DefaultStreamLoader) used Uri for resolve references.In this case format of correct reference is:
file://path.to.my.yaml, but it's wrong for OpenAPI.And it's not that I expect too.
I have written replacement for
DefaultStreamLoader:And use it as follows:
I expected that Micosoft.OpenAPI will resolve all JSON references and inline all files into the one document.
But now the feature works wired.
I found the great tutorial by specification splitting and I can do nothing of that.
In the current implementation you can reference only structural elements of OpenAPI. For example: schema, response, etc.
And furthermore, you need to add a header to each of secondary files:
And another bug
The reference will be replaced to the reference to the primary file, but the schema doesn't copy to primary file.
darrelmiller commentedon May 30, 2021
Thanks for the additional info, this will be very helpful.
Supporting of referencing external fragments will be supported. However, you can only use $ref to replace referenceable components, not just any arbitrary chunk of JSON. If you want to reference arbitrary chunks of JSON you will need to run the file through a pre-processor.
Inlining is not supposed to happen automatically. There is a setting in the writer if you want to inline on write.
OpenAPI.NET/src/Microsoft.OpenApi.Tool/OpenApiService.cs
Line 73 in a008173
V0v1kkk commentedon May 31, 2021
Thanks Darrel (@darrelmiller) !
But
ReferenceInlineSettingthere are only three values:DoNotInlineReferences,InlineLocalReferencesandInlineAllReferences.In my case I want to inline only remote references. 😒
Well, to be quite honest, I just want to split a huge specification (~34k lines in yaml) to smaller parts.
And merge the parts together on preprocessing without losing any information (names of schemes for example).
The bug that I mentioned above can be even helpful.
If I copy all parts from secondary OpenApiDocuments to primary, references will be fix automatically. 😆
Cooksauce commentedon May 31, 2021
Darrel (@darrelmiller) Thanks for taking a look. To be clear, I'm not concerned even whether or not it can resolve the reference, I just need it to be able to read the $ref string without throwing an error which prevents other properties from being parsed.
Additionally, it's worth noting that
$ref: path.to.my.yaml#/does work, while$ref: path.to.my.yamldoes not. IMO, the#/part should not be required in order for parsing to work (since they mean the same thing essentially). You can see why that happens here. I have have the reader settings set toReferenceResolutionSetting.DoNotResolveReferences. I've tried both with the latest stable nuget release and the latest preview.peteraritchie commentedon Jun 16, 2021
I think this is related:
textends up being"externalSchema.json#//externalPathSegment1/externalPathSegment2/externalPathSegment3"Note the two slashes
//after the fragment indicator (#).peteraritchie commentedon Jun 16, 2021
Interestingly, if I use realistic path segments (i.e:)
new OpenApiV3VersionService() .ConvertToOpenApiReference( "externalSchema.json#/components/schemas/Pet", null) .ReferenceV3...I get
"common.yaml#/Pet"But serializing that reference ends up with
"$ref: common.yaml#/Pet"(vianew OpenApiV3VersionService() .ConvertToOpenApiReference( "common.yaml#/components/schemas/Pet", null) .SerializeAsYaml(OpenApiSpecVersion.OpenApi3_0)That is, processing
"externalSchema.json#/components/schemas/Pet"when serialized back to a file becomes"$ref: common.yaml#/Pet"this is with vNext
Support loading an OpenApiDocument with remote references.
Merge pull request #619 from /issues/603