Repository navigation
Description
The QA team here is asking me about the validation of Boolean input values.
I need to give them an answer.
The GraphQL specification for input Boolean and String values states:
Reference: http://facebook.github.io/graphql/October2016/#sec-Boolean-Value
2.9.3 Boolean Value
BooleanValue: one of
true false
The two keywords true and false represent the two boolean values.
2.9.4 String Value
StringValue ::
""
" StringCharacter list "
[excerpted]
Strings are sequences of characters wrapped in double‐quotes ("). (ex. "Hello World").
White space and other otherwise‐ignored characters are significant within a string value.
Is this the reference specification used for the graphql-java implementation?
The same or a similar issue was recently addressed in graphql-js here: graphql/graphql-js#771
Example follows. The output is generated by the graphql java 6.0 release.
Schema Definition:
schema { query : Query }
type Query { ping ( input : Param ! ) : Ping }
input Param {
s : String !
b : Boolean !
}
type Ping {
s : String
b : Boolean
}
Sample requests and program output:
R-1:
{
"query" : "query ( $input : Param ! ) { ping ( input : $input ) { s b }}"
"variables" : {
"input" : {
"s" : 0
"b" : 0
}
}
}
Output: {"data":{"ping":{"s":"0","b":false}}}
R-2:
{
"query" : "query ( $input : Param ! ) { ping ( input : $input ) { s b }}"
"variables" : {
"input" : {
"s" : 123345,
"b" : 123345
}
}
}
Output: {"data":{"ping":{"s":"12345","b":true}}}
R-3:
{
"query" : "query ( $input : Param ! ) { ping ( input : $input ) { s b } }"
"variables" : {
"input" : {
"s" : "Atlanta",
"b" : "Atlanta"
}
}
}
Output: {"data":{"ping":{"s":"Atlanta","b":false}}}
R-4:
{
"query" : "query ( $input : Param ! ) { ping ( input : $input ) { s b } }"
"variables" : {
"input" : {
"s" : [1,2,3],
"b" : "Atlanta"
}
}
}
Output: {"data":{"ping":{"s":"[1, 2, 3]","b":false}}}
R-5:
{
"query" : "query ( $input : Param ! ) { ping ( input : $input ) { s b } }"
"variables" : {
"input" : {
"s" : [1,2,3],
"b" : [1,2,3]
}
}
}
Output: {"errors":[{"message":"Invalid input '[1, 2, 3]' for Boolean"}]}
Output Expectations:
R-1, R-2 should return an error for either s or b because the value of s is not quoted and the value of b is neither true nor false.
R-3 should return return an error for b because the value of b is neither true nor false.
R-4 should return an error for s because the value of s is not quoted
R-5 is ok because an error is returned for b as expected; but depending on the order of validation an error for s might be returned instead.
The Java 8 Program used for testing:
package graphqltest;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.module.SimpleModule;
import graphql.ExecutionInput;
import graphql.ExecutionResult;
import graphql.GraphQL;
import graphql.schema.GraphQLSchema;
import graphql.schema.idl.RuntimeWiring;
import graphql.schema.idl.SchemaGenerator;
import graphql.schema.idl.SchemaParser;
import graphql.schema.idl.TypeDefinitionRegistry;
import graphql.schema.idl.TypeInfo;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
public class GraphQLTest {
public static void main(String[] args) {
SchemaParser parser = new SchemaParser();
SchemaGenerator generator = new SchemaGenerator();
TypeDefinitionRegistry registry = parser.parse("schema { query : Query } type Query { ping ( input : Param ! ) : Ping } input Param { s : String ! b : Boolean ! } type Ping{ s : String b : Boolean }");
RuntimeWiring wiring = RuntimeWiring.newRuntimeWiring().type("Query", wires -> wires.dataFetcher("ping", env -> {
if (env.containsArgument("input")) { return env.getArgument("input"); }
return null;
})).build();
GraphQLSchema schema = generator.makeExecutableSchema(registry, wiring);
GraphQL gql = GraphQL.newGraphQL(schema).build();
ObjectMapper mapper = new ObjectMapper();
SimpleModule mod = new SimpleModule();
mod.addSerializer(TypeInfo.class, new JsonSerializer<TypeInfo>() {
@Override
public void serialize(TypeInfo t, JsonGenerator jg, SerializerProvider sp) throws IOException, JsonProcessingException { jg.writeString(t.toString()); }
});
mapper.registerModule(mod);
HashMapTypeReference typeRef = new HashMapTypeReference();
String[] requests = new String[]{
"{\"query\":\"query ( $input : Param ! ) { ping ( input : $input ) { s b }}\",\"variables\":{\"input\":{\"s\":0,\"b\":0}}}", // R-1
"{\"query\":\"query ( $input : Param ! ) { ping ( input : $input ) { s b }}\",\"variables\":{\"input\":{\"s\":12345,\"b\":12345}}}", // R-2
"{\"query\":\"query ( $input : Param ! ) { ping ( input : $input ) { s b }}\",\"variables\":{\"input\":{\"s\":\"Atlanta\",\"b\":\"Atlanta\"}}}", // R-3
"{\"query\":\"query ( $input : Param ! ) { ping ( input : $input ) { s b }}\",\"variables\":{\"input\":{\"s\":[1,2,3],\"b\":\"Atlanta\"}}}", // R-4
"{\"query\":\"query ( $input : Param ! ) { ping ( input : $input ) { s b }}\",\"variables\":{\"input\":{\"s\":[1,2,3],\"b\":[1,2,3]}}}" // R-5
};
for (String request : requests) {
try {
Map<String, Object> input = mapper.readValue(request, typeRef);
String query = (String) input.get("query");
Map<String, Object> variables = (Map<String, Object>) input.get("variables");
ExecutionResult result = gql.execute(ExecutionInput.newExecutionInput().query(query).variables(variables).build());
System.out.println("----------");
System.out.println("Request: " + request);
System.out.println(" Output: " + mapper.writeValueAsString(result.toSpecification()));
} catch (Exception ex) {
System.out.println(ex);
}
}
System.out.append("----------");
}
@SuppressWarnings("PublicInnerClass")
public static class HashMapTypeReference extends TypeReference<HashMap<String, Object>> { }
}
Libraries:
antlr-runtime-4.7.jar
graphql-java-6.0.jar
jackson-annotations-2.9.2.jar
jackson-core-2.9.2.jar
jackson-databind-2.9.2.jar
java-dataloader-2.0.1.jar
log4j-api-2.8.2.jar
log4j-core-2.8.2.jar
log4j-slf4j-impl-2.8.2.jar
slf4j-api-1.7.25.jar
Activity
bbakerman commentedon Dec 17, 2017
Thanks you for this detailed issue.
The graphql-java project aims to follow the graphql spec as much as possible. It was originally a port from the graphql-js reference implementation and we often use graphql-js behaviour as a "tie breaker" in expected behavior.
At first glance your Rn expectations look right however we will write them up in test form and investigate them. The Scalars.GraphqlString and Scalars.GraphqlBoolean are responsible for co-ercing values and are likely to be the place to fix potentially.
Given that they are only fixed on graphql-js master 10 days ago then its likely they same bugs are in the graphql-java port.
Once again thanks for reporting in such detail. A Pull Request with groovy spock tests outline expectations would be a great way to report such things in the future.
bbakerman commentedon Dec 17, 2017
Looking further into this it is indeed the
Scalar.XXX.parseValuemethod that is the source of the action.The graphql spec says
http://facebook.github.io/graphql/October2016/#sec-String
So reading that the input object to a Scalar.GraphqlString should only be a valid UTF-8 string.
However the graphql-js (reference implementation) has the following as its implementation
So it will take any object value (other than a list) and use JavaScript co-ercion to make it a String. so number input, boolean input and so on.
The graphql-java is also currently very liberal
So the dilema we now have is that the spec says one thing but the reference implementation does another.
So out there in graphql production land, a
numbervariable input to a String scalar type will succeed but the spec says it shouldnt. This expectation versus specification is what I am struggling with here.Mark-K commentedon Dec 18, 2017
Thank you for looking into this very difficult and potentially contentious issue.
I will look into how to do the pull request and making groovy spock tests for reporting future issues. Thank you for the suggestion.
My thinking is that when the input data is JSON, the Map<String, Object> received as Input should have already been parsed by the JSON library's (such as Jackson) string to Map<String, Object> parser and converted into the Java type as specified in the input data and JSON value conversion rules. So in the case of correct value syntax, there should be no need for value coercion.
For example:
JSON:
{ "s": "s", "b": true, "n": 123 }should be parsed into
Java Map<String, Object>
{ "s": String("s"), "b": Boolean.TRUE, "n": Integer(123) }As a result, the graphql-java library could require that the values of built-in scalars in the Input Map be the type specified by the schema, since the built-in scalars are basically the same as those defined by the JSON format. Though there may need to be some leniency with numeric type conversions such as up casting (Int -> Long) when necessary.
This would mean that the code that prepares the variable map must do "the right thing".
But that might not be current practice or not always reasonable, so making "strict" a requirement could break existing code and/or make parsing other data formats much more difficult. (i.e. web form data, Properties...)
My suggestion, would be (if the code will be modified) if it is feasible to make strict and lenient input validation selectable (with the default being lenient perhaps so as not to break anyone's hard work). When explicitly set to "strict", values in the input map must be the type specified by the schema without coercion. The "lenient" selection would provide the current (best effort to make sense out of the input) behavior.
This could be implemented in any of several different ways, for example:
mikeparisstuff commentedon Dec 19, 2017
I believe I've stumbled upon this issue as well and as an aside would also love to see the validation system become more pluggable in the future.
Given how the spec (http://facebook.github.io/graphql/October2016/#sec-String-Value) defines a string value I would think the implementation should validate that things that are being used as strings are indeed strings. I would expect the query in the snippet below to fail at validation time because I am trying to use a variable of type ID! for an argument of type String!.
Curious what paths forward there are to fix this.
Here is a snippet that isolates the issue.
Mark-K commentedon Dec 20, 2017
Michael,
Thanks for weighing in on this issue. You are reporting on a slightly different area than what I originally reported. My issue is with how the values of input variables are parsed (actually coerced) and validated. You are having a problem with how the query string itself is validated. This might deserve its own independent issue. But, having said that I have also seen (been surprised by) this behavior too.
Oops! Sent type X in the query when type Y was required. Hahaha...
Instead of an exception, an error with a more meaningful message would allow a client developer to understand/find/fix the mistake faster.
Input variable definition parsing needs to take the following issues into account (at a minimum).
In most cases the library responds correctly; but, in some cases an exception gets thrown.
The AssertionException is troublesome because it propagates all the way back to the application. If not handled by the application, no response is produced and the application unexpectedly terminates.
When the application is responding to client input, unexpected termination is not a very polite way to say you messed up.
Example 1 [OK]:
The parameter name is misspelled in the query input variable definition. An error message describing the problem is returned to the client. Basically variable str wasn't found but did find an unused variable star in the input variables.
Request: query Get($star: String!) { hello(str: $str) }
Output: {"errors":[{"message":"Validation error of type UndefinedVariable: Undefined variable str","locations":[{"line":1,"column":40}]},{"message":"Validation error of type UnusedVariable: Unused variable star","locations":[{"line":1,"column":11}]}]}
Example2 [OK]:
The query is okay but the variable name in the variables map is misspelled. An error message is returned which when translated says that variable str was not found in the variables map.
Request: query Get($str: String!) { hello(str: $str) }
Variables: new HashMap<String, Object>() {{ put("star", 1); }}
Output: {"errors":[{"message":"Variable 'str' has coerced Null value for NonNull type 'String!'"}]}
Example3 [OK]:
The input variable definition makes the str variable optional where as the schema says that its required. In this case as well, an error is returned to the client.
Request: query Get($str: String) { hello(str: $str) }
Output: {"errors":[{"message":"Validation error of type VariableTypeMismatch: Variable type doesn't match","locations":[{"line":1,"column":38}]}]}
In most cases the graphql library's response is correct.
For Schema:
schema { query: Query }
type Query { hello ( str: String!) : String }
input MyInput { str: String ! }
However, the library response is different depending on the input query string when the input variable type definition differs from the type specified by the schema. Testing results below show that Boolean returns an error message. But, 'Int!' throws an exception and 'Int' returns an error message with graphql claiming that 'Int' is an unknown type. Looking at the results below, when the input variable is a required String, an assertion exception will occur when other built-in scalar types Int, Long, ID (types other than Boolean) are specified.
I believe that I have also experienced the assertion exception when the required input type is user defined and a different user defined input type was specified in the input variable definition but have no evidence and didn't reproduce that case this time so maybe I am wrong and it was actually an error message instead.
Test Results:
Query: query Get( $str : String ) { hello (str: $str) }
Response: {"errors":[{"message":"Validation error of type VariableTypeMismatch: Variable type doesn't match","locations":[{"line":1,"column":38}]}]}
Query: query Get( $str : Boolean! ) { hello (str: $str) }
Response: {"errors":[{"message":"Validation error of type VariableTypeMismatch: Variable type doesn't match","locations":[{"line":1,"column":40}]}]}
Query: query Get($str: Boolean) { hello(str: $str) }
Response: {"errors":[{"message":"Validation error of type VariableTypeMismatch: Variable type doesn't match","locations":[{"line":1,"column":39}]}]}
Query: query Get( $str : Int! ) { hello (str: $str) }
Response: graphql.AssertException: wrappedType can't be null
Query: query Get($str: Int) { hello(str: $str) }
Response: {"errors":[{"message":"Validation error of type UnknownType: Unknown type Int","locations":[{"line":1,"column":17}]}]}
Query: query Get($str: MyInput) { hello(str: $str) }
Response: {"errors":[{"message":"Validation error of type VariableTypeMismatch: Variable type doesn't match","locations":[{"line":1,"column":39}]}]}
Query: query Get($str: MyInput!) { hello(str: $str) }
Response: {"errors":[{"message":"Validation error of type VariableTypeMismatch: Variable type doesn't match","locations":[{"line":1,"column":40}]}]}
Here is a copy of the stack trace if that is helpful. It was produced by graphql-java-6.0.jar
andimarek commentedon Jul 19, 2019
closing this fairly old issue now . Please open a new issue against the current graphql java version if there is still a problem with input coercion.
I am sorry that we didn't address this topic in time.