You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
DataFetchingFieldSelectionSet updates in 16.1 can't handle recursive schema properly and causing slowness #2185
Describe the bug
We are upgrading our library dependency from 15.0 to 16.1 recently and found that DataFetchingFieldSelectionSet updated at #2079 has a regression in performance making our HTTP request 10+ times longer.
We use Java Flight Recorder to narrow down the root cause and found that it is because the HashMap.resize() in NormalizedQueryTreeFactory.createNormalizedQueryImpl.
Below screenshots show that the normalizedFieldToMergedField has 1.29 M entries and other local variables here also extremely large in size.
To Reproduce
Please provide a code example or even better a test to reproduce the bug.
changed the title [-]DataFetchingFieldSelectionSet updates in 16.0 can't handle recursive schema properly and causing slowness[/-][+]DataFetchingFieldSelectionSet updates in 16.1 can't handle recursive schema properly and causing slowness[/+]on Feb 4, 2021
This time I notice that the selectionSet is being computed lazily. Remove usage of selection set makes performance back to normal.
About the Anonymize schema and query, the transformed schema has many validation errors (right) while the original one (left) has no validation error. I will take a look why but it seems that Schema Transformer doesn't handle certain edge cases.
@ntf in general we are aware that selectionSet is more compute intense now. That is the price we paid to be able to handle general lookups (including aliases and interfaces/unions).
The Anonymizer is a quite new (advanced) tool. It would be great to make it work with your schema. If you would be comfortable to share it in private you can reach me at http://twitter.com/andimarek
Commentary on this issue. Looking into the map usage here we have
Map<Field, List<NormalizedField>> fieldToNormalizedField = new LinkedHashMap<>();
Map<NormalizedField, MergedField> normalizedFieldToMergedField = new LinkedHashMap<>();
Map<FieldCoordinates, List<NormalizedField>> coordinatesToNormalizedFields = new LinkedHashMap<>();
at the top level of the processing. This is building out a map of all the NF -> to other things.
In a large query this could indeed blow out with re-sizes since there could be lots of Fields.
I cant see how the query listed above can cause this because its not very big. It has at most 6 fields. So this is making me wonder wether the FlightRecorder data is matching this query or not.
But I havent run this up in a debugger etc.. so I cant say fo certain.
One thought I have is that IF the validation code (which walks the query tree before hand - was to count how many nodes are present - then we could hint to later code to size structures better)
And yes I know there is the much bigger idea of building the NF data at the same time as validation but thats much more work
@andimarek - I tried to run up this schema from above. I think there might be a problem in the anonymizer code regarding interfaces and required fields etc...
The interface type 'Interface1' [@26:1] does not have a field 'field13' required via interface 'Interface3' [@47:1]],
The interface type 'Interface4' [@51:1] does not have a field 'field13' required via interface 'Interface3' [@47:1],
The interface type 'Interface5' [@63:1] does not have a field 'field13' required via interface 'Interface3' [@47:1],
The object type 'Object1' [@86:1] does not have a field 'field25' required via interface 'Interface4' [@51:1],
The object type 'Object1' [@86:1] does not have a field 'field13' required via interface 'Interface3' [@47:1],
The object type 'Object11' [@110:1] does not have a field 'field13' required via interface 'Interface3' [@47:1],
The object type 'Object12' [@123:1] does not have a field 'field13' required via interface 'Interface3' [@47:1],
The object type 'Object13' [@136:1] does not have a field 'field13' required via interface 'Interface3' [@47:1],
The object type 'Object14' [@148:1] does not have a field 'field13' required via interface 'Interface3' [@47:1], The object type 'Object15' [@159:1] does not have a field 'field13' required via interface 'Interface3' [@47:1], The object type 'Object18' [@180:1] does not have a field 'field45' required via interface 'Interface7' [@79:1], The object type 'Object18' [@180:1] does not have a field 'field46' required via interface 'Interface7' [@79:1], The object type 'Object18' [@180:1] does not have a field 'field47' required via interface 'Interface7' [@79:1], The object type 'Object18' [@180:1] does not have a field 'field13' required via interface 'Interface3' [@47:1], The object type 'Object19' [@193:1] does not have a field 'field45' required via interface 'Interface7' [@79:1], The object type 'Object19' [@193:1] does not have a field 'field46' required via interface 'Interface7' [@79:1], The object type 'Object19' [@193:1] does not have a field 'field47' required via interface 'Interface7' [@79:1], The object type 'Object19' [@193:1] does not have a field 'field13' required via interface 'Interface3' [@47:1], The object type 'Object20' [@211:1] does not have a field 'field45' required via interface 'Interface7' [@79:1], The object type 'Object20' [@211:1] does not have a field 'field46' required via interface 'Interface7' [@79:1], The object type 'Object20' [@211:1] does not have a field 'field47' required via interface 'Interface7' [@79:1],
The schema is not valid because of all the inheritance and fields not existing as required bu that inheritance.
@andimarek , thanks for the patch. below output is based on version: 2021-02-13T02-14-19-a9de9cec
errors: (24 of them)
graphql.schema.validation.InvalidSchemaException: invalid schema:
interface type 'Interface1' does not implement interface 'Interface3' because field 'field13' is missing
interface type 'Interface4' does not implement interface 'Interface3' because field 'field13' is missing
interface type 'Interface5' does not implement interface 'Interface3' because field 'field13' is missing
object type 'Object1' does not implement interface 'Interface4' because field 'field25' is missing
object type 'Object1' does not implement interface 'Interface3' because field 'field13' is missing
object type 'Object11' does not implement interface 'Interface3' because field 'field13' is missing
object type 'Object12' does not implement interface 'Interface3' because field 'field13' is missing
object type 'Object13' does not implement interface 'Interface3' because field 'field13' is missing
object type 'Object14' does not implement interface 'Interface3' because field 'field13' is missing
object type 'Object15' does not implement interface 'Interface3' because field 'field13' is missing
object type 'Object18' does not implement interface 'Interface7' because field 'field13' is missing
object type 'Object18' does not implement interface 'Interface3' because field 'field13' is missing
object type 'Object19' does not implement interface 'Interface7' because field 'field13' is missing
object type 'Object19' does not implement interface 'Interface3' because field 'field13' is missing
object type 'Object20' does not implement interface 'Interface7' because field 'field13' is missing
object type 'Object20' does not implement interface 'Interface3' because field 'field13' is missing
object type 'Object21' does not implement interface 'Interface7' because field 'field13' is missing
object type 'Object21' does not implement interface 'Interface3' because field 'field13' is missing
object type 'Object22' does not implement interface 'Interface2' because field 'field1' is missing
object type 'Object30' does not implement interface 'Interface3' because field 'field13' is missing
object type 'Object32' does not implement interface 'Interface3' because field 'field13' is missing
object type 'Object35' does not implement interface 'Interface3' because field 'field13' is missing
object type 'Object9' does not implement interface 'Interface4' because field 'field25' is missing
object type 'Object9' does not implement interface 'Interface3' because field 'field13' is missing
schema {
query: Object1
}
"Directs the executor to include this field or fragment only when the `if` argument is true"directive@include(
"Included when true."if: Boolean!
) onFIELD | FRAGMENT_SPREAD | INLINE_FRAGMENT"Directs the executor to skip this field or fragment when the `if`'argument is true."directive@skip(
"Skipped when true."if: Boolean!
) onFIELD | FRAGMENT_SPREAD | INLINE_FRAGMENTdirective@deprecated(reason: String) onFIELD_DEFINITION | ENUM_VALUE"Exposes a URL that specifies the behaviour of this scalar."directive@specifiedBy(
"The URL that specifies the behaviour of this scalar."url: String!
) onSCALARinterfaceInterface1implementsInterface2&Interface3 {
field2: ID!
}
interfaceInterface2 {
field2: ID!
}
interfaceInterface3 {
field2: ID!
}
typeObject1 {
field1: Object2
}
typeObject2implementsInterface1&Interface2&Interface3 {
field2: ID!
}
I can see that my fix worked only partially. Thanks to your schema I just merged another bugfix #2202 which hopefully should fix the problem for good 🤞
Please try again ... thanks a lot @ntf
@ntf in general we are aware that selectionSet is more compute intense now. That is the price we paid to be able to handle general lookups (including aliases and interfaces/unions).
Can you be more specific about the gains, esp. related to my question here: #2331 (is that supposed to work)?
I think you are "overnormalizing". In reality only a small subset of all possible relationships will be materialized in (almost) any recursive schema, yet you are materializing all possibilities upfront. I think it would be a lot better if you didn't ... You need to let us, users of graphql-java take our responsibility for knowing what to fetch (in fact, overnormalizing it makes it harder, I had to implement code to perform the inverse) and then you can lazily get only the stuff you discover in the actual fetched data ...
... and to qualify what makes it harder... my type hierarchies map to DB tables eventually. They include abstract/base types too with tables that contain data for these abstract/base types. By exploding all possible relationships you make it hard for me to discover that it really is just that column from that table in the end because it may be listed hundreds of times for different types. I have to walk my model back up to discover ... oh ... these hundred selected fields are really just this one here, from this type, so that I could build efficient queries.
Hi @ntf and @aleksandarsusnjar : please have a look at this PR #2345 which aims to solve this issue here. The plan is to get this PR merged soon. It would be great if you could provide early feedback if this solves your problem. Thanks
It seems the DataFetchingEnvironment#selectionSet() now resolves fields lazily upon accessing environment.getSelectionSet().getFields() and that finishes instantly.
I verified that this fixed my issue originally raised on Feb.
Edit: Got an NPE "java.lang.NullPointerException: null value in entry: viewAs=null" for the below
typeXXX {
owners(viewAs: ID = null): YYYConnection
}
Activity
[-]DataFetchingFieldSelectionSet updates in 16.0 can't handle recursive schema properly and causing slowness[/-][+]DataFetchingFieldSelectionSet updates in 16.1 can't handle recursive schema properly and causing slowness[/+]andimarek commentedon Feb 4, 2021
@ntf thanks for reporting.
Please make sure you are using 16.2: we fixed a cache in 16.2 regarding NormalizedTree.
If this doesn't fix it: please provide the query + schema you are using so that we can reproduce/benchmark it.
If you are not feeling comfortable to post the query + schema have a look at the Anonymizer class we recently added: https://github.com/graphql-java/graphql-java/blob/master/src/main/java/graphql/util/Anonymizer.java
and also the corresponding test class
https://github.com/graphql-java/graphql-java/blob/master/src/test/groovy/graphql/util/AnonymizerTest.groovy
ntf commentedon Feb 5, 2021
16.2 took less time to compute 70% vs 63% .
This time I notice that the
selectionSetis being computed lazily. Remove usage of selection set makes performance back to normal.About the Anonymize schema and query, the transformed schema has many validation errors (right) while the original one (left) has no validation error. I will take a look why but it seems that Schema Transformer doesn't handle certain edge cases.

andimarek commentedon Feb 5, 2021
@ntf in general we are aware that
selectionSetis more compute intense now. That is the price we paid to be able to handle general lookups (including aliases and interfaces/unions).The Anonymizer is a quite new (advanced) tool. It would be great to make it work with your schema. If you would be comfortable to share it in private you can reach me at http://twitter.com/andimarek
ntf commentedon Feb 5, 2021
I use a debugger to print the transformed schema with the
SchemaPrinter. Hope this helps.andimarek commentedon Feb 5, 2021
thanks @ntf : we would also need the query to actually benchmark it. Or can you not produce the anomymized query because of the error?
ntf commentedon Feb 5, 2021
@andimarek
Actually I can by clearing the error on the debugger.
bbakerman commentedon Feb 8, 2021
Commentary on this issue. Looking into the map usage here we have
at the top level of the processing. This is building out a map of all the NF -> to other things.
In a large query this could indeed blow out with re-sizes since there could be lots of Fields.
I cant see how the query listed above can cause this because its not very big. It has at most 6 fields. So this is making me wonder wether the FlightRecorder data is matching this query or not.
But I havent run this up in a debugger etc.. so I cant say fo certain.
One thought I have is that IF the validation code (which walks the query tree before hand - was to count how many nodes are present - then we could hint to later code to size structures better)
And yes I know there is the much bigger idea of building the NF data at the same time as validation but thats much more work
ntf commentedon Feb 8, 2021
@bbakerman
How about this below ? I extracted from my frontend query to the backend.
query operation($var1:ID) {field113(argument55:$var1) {__typename ... on Interface4 {alias1:__typename field25} ...Fragment1 ...Fragment5 field13} field98 {field1 ...Fragment6}} fragment Fragment1 on Interface2 {alias2:__typename field1 field2 field3 {__typename field1 field2} ... on Interface5 {alias3:__typename field30 field31} ... on Interface1 {alias4:__typename field14} ...Fragment2 ...Fragment3} fragment Fragment4 on Interface2 {alias5:__typename field1 field2 field3 {__typename field1} ... on Interface4 {alias6:__typename field25 field28 {__typename field1}}} fragment Fragment5 on Interface2 {alias7:__typename field1 field2 ... on Interface4 {alias8:__typename field25} field3 {__typename field1 field2 ... on Interface4 {alias9:__typename field25} ...Fragment3} ... on Interface1 {alias10:__typename field15 field16 field18}} fragment Fragment6 on Interface5 {alias11:__typename field1} fragment Fragment2 on Interface5 {alias12:__typename ...Fragment3} fragment Fragment3 on Interface2 {alias13:__typename ...Fragment4}bbakerman commentedon Feb 8, 2021
@andimarek - I tried to run up this schema from above. I think there might be a problem in the anonymizer code regarding interfaces and required fields etc...
The schema is not valid because of all the inheritance and fields not existing as required bu that inheritance.
andimarek commentedon Feb 13, 2021
Hi @ntf : can you please the latest dev version of the anonymizer: I identified and fixed a bug. See #2201
ntf commentedon Feb 13, 2021
@andimarek , thanks for the patch. below output is based on version:
2021-02-13T02-14-19-a9de9cecerrors: (24 of them)
schema:
query:
andimarek commentedon Feb 13, 2021
Thanks @ntf, but exactly the same bug I fixed seems to be still present.
Can you confirm that your are using the version you mentioned?
If so, can you anonomyize the small Schema from this test: https://github.com/graphql-java/graphql-java/blob/master/src/test/groovy/graphql/util/AnonymizerTest.groovy#L550
Thanks for your help!
ntf commentedon Feb 13, 2021
input:
output:
andimarek commentedon Feb 13, 2021
I can see that my fix worked only partially. Thanks to your schema I just merged another bugfix #2202 which hopefully should fix the problem for good 🤞
Please try again ... thanks a lot @ntf
ntf commentedon Feb 14, 2021
@andimarek , congratulation! The fix #2202 works on my schema!
version: 2021-02-13T23-17-27-86627c27
query:
aleksandarsusnjar commentedon May 11, 2021
Seems related: #2309
ntf commentedon May 12, 2021
@aleksandarsusnjar I think it is related.
meanwhile I just downgraded to 15.0 so that I can continue to work on the project.
aleksandarsusnjar commentedon May 12, 2021
@andimarek: You said:
Can you be more specific about the gains, esp. related to my question here: #2331 (is that supposed to work)?
I think you are "overnormalizing". In reality only a small subset of all possible relationships will be materialized in (almost) any recursive schema, yet you are materializing all possibilities upfront. I think it would be a lot better if you didn't ... You need to let us, users of
graphql-javatake our responsibility for knowing what to fetch (in fact, overnormalizing it makes it harder, I had to implement code to perform the inverse) and then you can lazily get only the stuff you discover in the actual fetched data ...... and to qualify what makes it harder... my type hierarchies map to DB tables eventually. They include abstract/base types too with tables that contain data for these abstract/base types. By exploding all possible relationships you make it hard for me to discover that it really is just that column from that table in the end because it may be listed hundreds of times for different types. I have to walk my model back up to discover ... oh ... these hundred selected fields are really just this one here, from this type, so that I could build efficient queries.
andimarek commentedon May 19, 2021
Hi @ntf and @aleksandarsusnjar : please have a look at this PR #2345 which aims to solve this issue here. The plan is to get this PR merged soon. It would be great if you could provide early feedback if this solves your problem. Thanks
aleksandarsusnjar commentedon May 19, 2021
@andimarek Will do!
andimarek commentedon May 20, 2021
the PR is now merged and the dev build contains the improvements @ntf @aleksandarsusnjar
ntf commentedon May 20, 2021
Thanks, @andimarek . Which dev build contains the changes?
the latest one I can find on Maven central is https://search.maven.org/artifact/com.graphql-java/graphql-java/0.0.0-2021-05-17T23-49-29-83942204/jar
andimarek commentedon May 21, 2021
@ntf I found a better view on the available artefacts is this: https://repo1.maven.org/maven2/com/graphql-java/graphql-java/
There you can see a newer build: https://repo1.maven.org/maven2/com/graphql-java/graphql-java/0.0.0-2021-05-20T01-15-22-fdd8d1a7/
ntf commentedon Jun 28, 2021
Hi @andimarek, sorry for the delay.
It seems the DataFetchingEnvironment#selectionSet() now resolves fields lazily upon accessing
environment.getSelectionSet().getFields()and that finishes instantly.I verified that this fixed my issue originally raised on Feb.
Edit: Got an NPE "java.lang.NullPointerException: null value in entry: viewAs=null" for the below
graphql.com.google.common.collect.CollectPreconditions.checkEntryNotNull(CollectPreconditions.java:32)
....
graphql.com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:438)
graphql.normalized.NormalizedField$Builder.resolvedArguments(NormalizedField.java:310)