Skip to content

FastSchemaValidator - #4549

Open
xuorig wants to merge 2 commits into
graphql-java:masterfrom
xuorig:fast-schema-validator
Open

xuorig wants to merge 2 commits into
graphql-java:masterfrom
xuorig:fast-schema-validator

Conversation

@xuorig

@xuorig xuorig commented Nov 10, 2026

Copy link
Copy Markdown
Contributor

Introduces a FastSchemaValidator, which doesn't use SchemaTraverser. Running locally, I achieve ~20x speedup compared to SchemaValidator. I've chosen to introduce it as the default validator for the FastBuilder.

For correctness, with the help of Opus I've ran every single GraphQLSchema build in the test suite with both validators and they came back with the same set of errors. I added a new set of comparison tests too (similar to FastBuilder) for correctness and making sure they stay aligned.

A benchmark was also added.

Let me know how you want to handle this, we could:

  1. Make this one the default everywhere right now
  2. Make it the default in fast builder (CURRENT IMPLEMENTATION)
  3. Keep it outside both builders for now, can only be used "manually" through experimental API.

Local Results:

Setup: JMH 1.37, JDK 25.0.2 (OpenJDK 64-Bit Server VM) on my macOS machine. 2 forks, 2 warmup iterations of 5 s, 3 measurement iterations of 10 s, GC profiler on.

./gradlew jmh -PjmhInclude='SchemaValidatorBenchmark' -PjmhProfilers=gc

Benchmark                                                                             (schemaFile)  Mode  Cnt          Score         Error   Units
SchemaValidatorBenchmark.fastSchemaValidator                               large-schema-4.graphqls  avgt    6         15.579 ±       1.665   ms/op
SchemaValidatorBenchmark.fastSchemaValidator:gc.alloc.rate                 large-schema-4.graphqls  avgt    6        716.824 ±      72.763  MB/sec
SchemaValidatorBenchmark.fastSchemaValidator:gc.alloc.rate.norm            large-schema-4.graphqls  avgt    6   11696271.545 ±   10091.025    B/op
SchemaValidatorBenchmark.fastSchemaValidator:gc.count                      large-schema-4.graphqls  avgt    6         57.000                counts
SchemaValidatorBenchmark.fastSchemaValidator:gc.time                       large-schema-4.graphqls  avgt    6         43.000                    ms
SchemaValidatorBenchmark.fastSchemaValidator                     large-schema-federated-1.graphqls  avgt    6         16.900 ±       2.487   ms/op
SchemaValidatorBenchmark.fastSchemaValidator:gc.alloc.rate       large-schema-federated-1.graphqls  avgt    6        452.646 ±      62.060  MB/sec
SchemaValidatorBenchmark.fastSchemaValidator:gc.alloc.rate.norm  large-schema-federated-1.graphqls  avgt    6    8004115.706 ±   74511.963    B/op
SchemaValidatorBenchmark.fastSchemaValidator:gc.count            large-schema-federated-1.graphqls  avgt    6         26.000                counts
SchemaValidatorBenchmark.fastSchemaValidator:gc.time             large-schema-federated-1.graphqls  avgt    6        127.000                    ms
SchemaValidatorBenchmark.schemaValidator                                   large-schema-4.graphqls  avgt    6        383.427 ±      13.434   ms/op
SchemaValidatorBenchmark.schemaValidator:gc.alloc.rate                     large-schema-4.graphqls  avgt    6       1323.517 ±      52.054  MB/sec
SchemaValidatorBenchmark.schemaValidator:gc.alloc.rate.norm                large-schema-4.graphqls  avgt    6  532055193.398 ± 2804385.051    B/op
SchemaValidatorBenchmark.schemaValidator:gc.count                          large-schema-4.graphqls  avgt    6        256.000                counts
SchemaValidatorBenchmark.schemaValidator:gc.time                           large-schema-4.graphqls  avgt    6       9077.000                    ms
SchemaValidatorBenchmark.schemaValidator                         large-schema-federated-1.graphqls  avgt    6        309.378 ±      15.368   ms/op
SchemaValidatorBenchmark.schemaValidator:gc.alloc.rate           large-schema-federated-1.graphqls  avgt    6       1247.296 ±      69.868  MB/sec
SchemaValidatorBenchmark.schemaValidator:gc.alloc.rate.norm      large-schema-federated-1.graphqls  avgt    6  404519204.968 ± 3888410.861    B/op
SchemaValidatorBenchmark.schemaValidator:gc.count                large-schema-federated-1.graphqls  avgt    6        230.000                counts
SchemaValidatorBenchmark.schemaValidator:gc.time                 large-schema-federated-1.graphqls  avgt    6       6366.000                    ms

TL;DR 
┌──────────────────────────┬─────────────────┬─────────────────────┬─────────┬──────────────────────────┐
│          Schema          │ SchemaValidator │ FastSchemaValidator │ Speedup │ Memory allocated per run │
├──────────────────────────┼─────────────────┼─────────────────────┼─────────┼──────────────────────────┤
│ large-schema-4           │ 383.4 ± 13.4 ms │ 15.6 ± 1.7 ms       │ ~25×    │ 532 MB → 11.7 MB         │
├──────────────────────────┼─────────────────┼─────────────────────┼─────────┼──────────────────────────┤
│ large-schema-federated-1 │ 309.4 ± 15.4 ms │ 16.9 ± 2.5 ms       │ ~18×    │ 405 MB → 8.0 MB          │
└──────────────────────────┴─────────────────┴─────────────────────┴─────────┴──────────────────────────┘

@xuorig
xuorig force-pushed the fast-schema-validator branch from 6b2361e to 4762b7c Compare November 10, 2026 15:43
@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Test Report

Test Results

Java Version Total Passed Failed Errors Skipped
Java 17 6634 (+37 🟢) 6578 (+37 🟢) 0 (±0) 0 (±0) 56 (±0)
Java 21 6634 (+37 🟢) 6578 (+37 🟢) 0 (±0) 0 (±0) 56 (±0)
Java 25 6634 (+37 🟢) 6578 (+37 🟢) 0 (±0) 0 (±0) 56 (±0)
jcstress 32 (±0) 32 (±0) 0 (±0) 0 (±0) 0 (±0)
Total 19934 (+111 🟢) 19766 (+111 🟢) 0 (±0) 0 (±0) 168 (±0)

Code Coverage (Java 25)

Metric Covered Missed Coverage vs Master
Lines 30723 2959 91.2% +0.1% 🟢
Branches 9236 1440 86.5% +0.5% 🟢
Methods 8192 1158 87.6% ±0.0%

Changed Class Coverage (11 classes)

Class Line Branch Method
g.s.i.SchemaGeneratorHelper +0.6% 🟢 +1.1% 🟢 ±0.0%
g.s.ShallowTypeRefCollector +1.3% 🟢 +4.2% 🟢 ±0.0%
g.s.v.AppliedDirectiveArgumentsAreValid +2.6% 🟢 +4.5% 🟢 ±0.0%
g.s.v.AppliedDirectivesAreValid +4.3% 🟢 +10.0% 🟢 ±0.0%
g.s.v.DefaultValuesAreValid +10.0% 🟢 +12.5% 🟢 ±0.0%
g.s.v.InputValueValidator +100.0% 🟢 +100.0% 🟢 +100.0% 🟢
g.s.v.NoDefaultValueCircularRefs +10.3% 🟢 +18.3% 🟢 ±0.0%
g.s.v.NoUnbrokenInputCycles +2.9% 🟢 +5.6% 🟢 ±0.0%
g.s.v.SchemaValidationErrorCollector +22.2% 🟢 +50.0% 🟢 ±0.0%
g.s.v.SchemaValidationRule +100.0% 🟢 ±0.0% +100.0% 🟢
g.s.v.TypeAndFieldRule +16.1% 🟢 +22.9% 🟢 +10.5% 🟢

Full HTML report: build artifact jacoco-html-report

Updated: 2026-11-13 13:29:36 UTC

@andimarek

Copy link
Copy Markdown
Member

hey @xuorig ... thanks a lot.

We will continue to prioritize performance and also thanks to AI we are not so concerned anymore with more specialized code if it serves our goals. Hence I am leaning toward using the FastSchemaValidator everywhere and get rid of the other one.

cc @bbakerman your thoughts?

@xuorig
xuorig force-pushed the fast-schema-validator branch from 4762b7c to 0e8eb4d Compare November 11, 2026 13:43
@xuorig

xuorig commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor Author

We will continue to prioritize performance and also thanks to AI we are not so concerned anymore with more specialized code if it serves our goals. Hence I am leaning toward using the FastSchemaValidator everywhere and get rid of the other one.

Makes sense to me! I've updated the PR to replace the validator entirely. There was also another approach where we could have used FastSchemaValidator in the schema builders, but still leave SchemaValidator as a public API. getRules for example was removed here, which is a breaking change. If you were planning to cut a new major release that might be OK, but let me know, I can spin up the no-breaking-change version too.

Edit: Actually I forgot SchemaValidator is @Internal, so maybe that's fine

@xuorig
xuorig force-pushed the fast-schema-validator branch 2 times, most recently from 63782b7 to ed77804 Compare November 11, 2026 14:22
@bbakerman

Copy link
Copy Markdown
Member

hey @xuorig ... thanks a lot.

We will continue to prioritize performance and also thanks to AI we are not so concerned anymore with more specialized code if it serves our goals. Hence I am leaning toward using the FastSchemaValidator everywhere and get rid of the other one.

cc @bbakerman your thoughts?

If this is faster we want it - large schemas are a bottle neck in graphql-java - we are hitting that in Atlassian where schema loading is taking longer than we wish (with 35K types and 85K fields) =- getting 18x to 25x is a rare find

Given that this passes all tests and the tests havent been fundamentally changed - I think this should be the new way we validate.

@bbakerman bbakerman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel we should go with this approach

Thanks

@xuorig

xuorig commented Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

Local bench.

A bit of overhead due to calling methods on every elements no matter what, but still a huge improvement.

┌──────────────────────────┬──────────────────┬──────────────────┬─────────────────┐
│          Schema          │   single class   │ rules as classes │     master      │
├──────────────────────────┼──────────────────┼──────────────────┼─────────────────┤
│ large-schema-4           │ 15.6 ms, 11.7 MB │ 20.4 ms, 13.0 MB │ ~407 ms, 532 MB │
├──────────────────────────┼──────────────────┼──────────────────┼─────────────────┤
│ large-schema-federated-1 │ 17.0 ms, 8.5 MB  │ 25.0 ms, 10.3 MB │ ~323 ms, 398 MB │
└──────────────────────────┴──────────────────┴──────────────────┴─────────────────┘

@xuorig
xuorig force-pushed the fast-schema-validator branch 2 times, most recently from b6aae19 to 465e0fd Compare November 12, 2026 13:49
@xuorig

xuorig commented Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

Rebased due to the new DefaultValueDepth check that was added to master.

@xuorig
xuorig force-pushed the fast-schema-validator branch from 465e0fd to 899d448 Compare November 12, 2026 14:06
…erser walk

SchemaValidator now applies its rules by iterating the schema's types and
directive definitions directly, instead of running ten GraphQLTypeVisitor
rules over every schema element with SchemaTraverser. validateSchema keeps
the same signature; getRules() and the rule visitor classes are removed.

Behaviour differences:
- errors are reported grouped by type, so their order changes
- elements shared between several parents are checked under each of them,
  where the traversal only checked them under the first parent it reached
- schemas whose programmatic default values form a cycle now fail with an
  InvalidSchemaException instead of a StackOverflowError

On large-schema-4 validation drops from ~407ms to ~16ms and allocation from
~532MB to ~12MB per run (SchemaValidatorBenchmark).
SchemaValidator keeps the single loop over types and directive definitions
and calls the hooks of a SchemaValidationRule per rule class. Each
validateSchema call creates its own rule instances, so per-validation state
stays in final fields. InputValueValidator holds the value coercion shared
by DefaultValuesAreValid and AppliedDirectiveArgumentsAreValid.
@xuorig
xuorig force-pushed the fast-schema-validator branch from 899d448 to 56c99ea Compare November 13, 2026 13:20
@xuorig

xuorig commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

Rebased again!

@andimarek

Copy link
Copy Markdown
Member

@xuorig my AI said:

[P2] Arguments inside legacy directive applications lose validation. The new loop checks their supplied values, but skips other argument rules. An Int argument with default "bad", or an undefined directive attached to that argument, now passes schema construction. Both are rejected on master.

[P2] Schema-level legacy directives are omitted. The validator iterates types and directive definitions, but never schema.getSchemaDirectives(). An invalid argument default on a schema-level directive now passes construction.

We have some other PRs which overlap slightly with that .. .I might push changes to this branch

@xuorig

xuorig commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

@andimarek please feel free yes, thanks.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants