You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Include a caching document provider implementation by default #4107
Caching the parsing and validation of a document can save significant latency.
For very large operations, and tools like Relay produce very large operations, the parse and validate latency can be upwards of 15-20ms (p99)
By caching this step you can amortize the cost over many requests and hence cut nearly all of this latency out.
The graphql.execution.preparsed.PreparsedDocumentProvider is the interface that allows you to build a caching document mechanism but out of the box graphql-java does not provide a working implementation
I propose that we
Create a public class CachingDocumentProvider implements PreparsedDocumentProvider
That is used Caffeine as the cache
Provide a GraphQLUnusualConfiguration mechanism to to tune the cache
The implication here is that https://github.com/ben-manes/caffeine would become a dependency (shaded) and we have mostly aimed for low to zero dependencies but I think the juice is worth the squeeze here. All operations will run faster with a cache in place.
I dont know right now of the implications for library frameworks such as Spring GraphQL. Do they have one themselves, is it always on? Hence this issue to stand as a discussion mechanism
What are the typical challenges for implementing such an interface?
I noticed one implementation in DGS and it doesn't look too extensive. I'm wondering if shading (extra jar size, extra maintenance work for the team) is the best solution, since applications might want to use their custom version of caffeine, with custom options and maybe share the allocated memory with other bits of the application?
Maybe a sample implementation in the reference docs and more guidance around persisted queries would be more valuable to the community.
What is it specifically from caffeine that we would leverage?
Could we not have a default implementation that ist just a hashmap and people could adapt caffeine to it themself?
The reason for Caffeine is that we can then release graphql-java (eventually) with this caching turned on. This will make ALL graphql-java implementations faster - a lot laster - 5 / 10/ 20 ms faster
We cant ship with a HashMap because of its unbounded nature. There must be a LRU type cache in place so we dont by default blow memory limits. And this is what leads us to Caffeine. Its a world class cache, it can efficiently evict entries and hence means we can have this on by default but be sensible in memory usage
Other alternatives:
Release it as extra maven artifact
Yes considered that but then its can be on by default. Hence the 5 / 10/ 20 ms faster savings are not on by default.
multi module jar (no idea how/if that would work)
No idea either
have a runtime reflection detection like spring that leverages caffeine if in the classpath
This is not crazy however I would like to put the configuration of the cache into the graphql.GraphQLUnusualConfiguration and hence expose direct API level for configuring the cache
This is some early benchmarking from having the caching in place. As you can see the larger the query the great the payback. This is expected since the time for parsing text is directly correlated with the size of the text.
So caching query parsing and validation pays off as queries become larger and with tools like Relay "computer generating queries" they can get very large easily.
Hey folks, I think frameworks (e.g., Spring/DGS) should provide this instead.
It's easy to provide for a framework, and that way graphql-java doesn't need to pull in a new external dependency.
One of the strengths of GraphQL-Java is that it's a foundational library without many dependencies, configurations, or opinions. Adding caching by default gets into a gray area without much benefit because frameworks can already do this anyway.
That being said, I agree on the benefits of having a cache; we're seeing this in practice as well.
Hello, this issue has been inactive for 60 days, so we're marking it as stale. If you would like to continue this discussion, please comment within the next 30 days or we'll close the issue.
Activity
bbakerman commentedon Sep 12, 2025
Caffeine has the following dependencies should we choose to include it as a dep
andimarek commentedon Sep 12, 2025
What is it specifically from caffeine that we would leverage?
Could we not have a default implementation that ist just a hashmap and people could adapt caffeine to it themself?
Other alternatives:
Release it as extra maven artifact
multi module jar (no idea how/if that would work)
have a runtime reflection detection like spring that leverages caffeine if in the classpath
bclozel commentedon Sep 12, 2025
What are the typical challenges for implementing such an interface?
I noticed one implementation in DGS and it doesn't look too extensive. I'm wondering if shading (extra jar size, extra maintenance work for the team) is the best solution, since applications might want to use their custom version of caffeine, with custom options and maybe share the allocated memory with other bits of the application?
Maybe a sample implementation in the reference docs and more guidance around persisted queries would be more valuable to the community.
bbakerman commentedon Sep 14, 2025
The reason for Caffeine is that we can then release graphql-java (eventually) with this caching turned on. This will make ALL graphql-java implementations faster - a lot laster - 5 / 10/ 20 ms faster
We cant ship with a HashMap because of its unbounded nature. There must be a LRU type cache in place so we dont by default blow memory limits. And this is what leads us to Caffeine. Its a world class cache, it can efficiently evict entries and hence means we can have this on by default but be sensible in memory usage
Yes considered that but then its can be on by default. Hence the 5 / 10/ 20 ms faster savings are not on by default.
No idea either
This is not crazy however I would like to put the configuration of the cache into the
graphql.GraphQLUnusualConfigurationand hence expose direct API level for configuring the cachebut perhaps we can look more info this. eg becomes a NO op if there if Caffeine is not present.
bbakerman commentedon Sep 21, 2025
This is some early benchmarking from having the caching in place. As you can see the larger the query the great the payback. This is expected since the time for parsing text is directly correlated with the size of the text.
So caching query parsing and validation pays off as queries become larger and with tools like Relay "computer generating queries" they can get very large easily.
paulbakker commentedon Oct 6, 2025
Hey folks, I think frameworks (e.g., Spring/DGS) should provide this instead.
It's easy to provide for a framework, and that way graphql-java doesn't need to pull in a new external dependency.
One of the strengths of GraphQL-Java is that it's a foundational library without many dependencies, configurations, or opinions. Adding caching by default gets into a gray area without much benefit because frameworks can already do this anyway.
That being said, I agree on the benefits of having a cache; we're seeing this in practice as well.
github-actions commentedon Dec 6, 2025
Hello, this issue has been inactive for 60 days, so we're marking it as stale. If you would like to continue this discussion, please comment within the next 30 days or we'll close the issue.