Skip to content

Is input validation too loose for String and Boolean? #865

Closed

Description

@Mark-K

The QA team here is asking me about the validation of Boolean input values.
I need to give them an answer.

The GraphQL specification for input Boolean and String values states:
Reference: http://facebook.github.io/graphql/October2016/#sec-Boolean-Value

2.9.3 Boolean Value
    BooleanValue: one of
       true    false
    The two keywords true and false represent the two boolean values.
2.9.4 String Value
    StringValue ::
        ""
        " StringCharacter list "
    [excerpted]
    Strings are sequences of characters wrapped in double‐quotes ("). (ex. "Hello World").
    White space and other otherwise‐ignored characters are significant within a string value.

Is this the reference specification used for the graphql-java implementation?

The same or a similar issue was recently addressed in graphql-js here: graphql/graphql-js#771

Example follows. The output is generated by the graphql java 6.0 release.
Schema Definition:

schema { query : Query }
type Query { ping ( input : Param ! ) : Ping }
input Param {
   s : String !
   b : Boolean !
}
type Ping {
   s : String
   b : Boolean
}

Sample requests and program output:

R-1:

{
    "query" : "query ( $input : Param ! ) { ping ( input : $input ) { s b }}"
    "variables" : {
        "input" : {
            "s" : 0
            "b" : 0
        }
    }
}

Output: {"data":{"ping":{"s":"0","b":false}}}

R-2:

{
    "query" : "query ( $input : Param ! ) { ping ( input : $input ) { s b }}"
    "variables" : {
        "input" : {
            "s" : 123345,
            "b" : 123345
        }
    }
}

Output: {"data":{"ping":{"s":"12345","b":true}}}

R-3:

{
    "query" : "query ( $input : Param ! ) { ping ( input : $input ) { s b } }"
    "variables" : {
        "input" : {
            "s" : "Atlanta",
            "b" : "Atlanta"
        }
    }
}

Output: {"data":{"ping":{"s":"Atlanta","b":false}}}

R-4:

{
    "query" : "query ( $input : Param ! ) { ping ( input : $input ) { s b } }"
    "variables" : {
        "input" : {
            "s" : [1,2,3],
            "b" : "Atlanta"
        }
    }
}

Output: {"data":{"ping":{"s":"[1, 2, 3]","b":false}}}

R-5:

{
    "query" : "query ( $input : Param ! ) { ping ( input : $input ) { s b } }"
    "variables" : {
        "input" : {
            "s" : [1,2,3],
            "b" : [1,2,3]
        }
    }
}

Output: {"errors":[{"message":"Invalid input '[1, 2, 3]' for Boolean"}]}

Output Expectations:
R-1, R-2 should return an error for either s or b because the value of s is not quoted and the value of b is neither true nor false.
R-3 should return return an error for b because the value of b is neither true nor false.
R-4 should return an error for s because the value of s is not quoted
R-5 is ok because an error is returned for b as expected; but depending on the order of validation an error for s might be returned instead.

The Java 8 Program used for testing:

package graphqltest;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.JsonSerializer;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.module.SimpleModule;
import graphql.ExecutionInput;
import graphql.ExecutionResult;
import graphql.GraphQL;
import graphql.schema.GraphQLSchema;
import graphql.schema.idl.RuntimeWiring;
import graphql.schema.idl.SchemaGenerator;
import graphql.schema.idl.SchemaParser;
import graphql.schema.idl.TypeDefinitionRegistry;
import graphql.schema.idl.TypeInfo;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
public class GraphQLTest {
    public static void main(String[] args) {
        SchemaParser parser = new SchemaParser();
        SchemaGenerator generator = new SchemaGenerator();
        TypeDefinitionRegistry registry = parser.parse("schema { query : Query } type Query { ping ( input : Param ! ) : Ping } input Param { s : String ! b : Boolean ! } type Ping{ s : String b : Boolean }");
        RuntimeWiring wiring = RuntimeWiring.newRuntimeWiring().type("Query", wires -> wires.dataFetcher("ping", env -> {
            if (env.containsArgument("input")) { return env.getArgument("input"); }
            return null;
        })).build();
        GraphQLSchema schema = generator.makeExecutableSchema(registry, wiring);
        GraphQL gql = GraphQL.newGraphQL(schema).build();
        ObjectMapper mapper = new ObjectMapper();
        SimpleModule mod = new SimpleModule();
        mod.addSerializer(TypeInfo.class, new JsonSerializer<TypeInfo>() {
            @Override
            public void serialize(TypeInfo t, JsonGenerator jg, SerializerProvider sp) throws IOException, JsonProcessingException { jg.writeString(t.toString()); }
        });
        mapper.registerModule(mod);
        HashMapTypeReference typeRef = new HashMapTypeReference();
        String[] requests = new String[]{
            "{\"query\":\"query ( $input : Param ! ) { ping ( input : $input ) { s b }}\",\"variables\":{\"input\":{\"s\":0,\"b\":0}}}",                        // R-1
            "{\"query\":\"query ( $input : Param ! ) { ping ( input : $input ) { s b }}\",\"variables\":{\"input\":{\"s\":12345,\"b\":12345}}}",                // R-2
            "{\"query\":\"query ( $input : Param ! ) { ping ( input : $input ) { s b }}\",\"variables\":{\"input\":{\"s\":\"Atlanta\",\"b\":\"Atlanta\"}}}",    // R-3
            "{\"query\":\"query ( $input : Param ! ) { ping ( input : $input ) { s b }}\",\"variables\":{\"input\":{\"s\":[1,2,3],\"b\":\"Atlanta\"}}}",        // R-4
            "{\"query\":\"query ( $input : Param ! ) { ping ( input : $input ) { s b }}\",\"variables\":{\"input\":{\"s\":[1,2,3],\"b\":[1,2,3]}}}"             // R-5
        };
        for (String request : requests) {
            try {
                Map<String, Object> input = mapper.readValue(request, typeRef);
                String query = (String) input.get("query");
                Map<String, Object> variables = (Map<String, Object>) input.get("variables");
                ExecutionResult result = gql.execute(ExecutionInput.newExecutionInput().query(query).variables(variables).build());
                System.out.println("----------");
                System.out.println("Request: " + request);
                System.out.println(" Output: " + mapper.writeValueAsString(result.toSpecification()));
            } catch (Exception ex) {
                System.out.println(ex);
            }
        }
        System.out.append("----------");
    }
    @SuppressWarnings("PublicInnerClass")
    public static class HashMapTypeReference extends TypeReference<HashMap<String, Object>> { }
}

Libraries:

antlr-runtime-4.7.jar
graphql-java-6.0.jar
jackson-annotations-2.9.2.jar
jackson-core-2.9.2.jar
jackson-databind-2.9.2.jar
java-dataloader-2.0.1.jar
log4j-api-2.8.2.jar
log4j-core-2.8.2.jar
log4j-slf4j-impl-2.8.2.jar
slf4j-api-1.7.25.jar

Activity

  1. bbakerman commented on Dec 17, 2017

    @bbakerman
    Member

    Thanks you for this detailed issue.

    Is this the reference specification used for the graphql-java implementation?

    The graphql-java project aims to follow the graphql spec as much as possible. It was originally a port from the graphql-js reference implementation and we often use graphql-js behaviour as a "tie breaker" in expected behavior.

    At first glance your Rn expectations look right however we will write them up in test form and investigate them. The Scalars.GraphqlString and Scalars.GraphqlBoolean are responsible for co-ercing values and are likely to be the place to fix potentially.

    Given that they are only fixed on graphql-js master 10 days ago then its likely they same bugs are in the graphql-java port.

    Once again thanks for reporting in such detail. A Pull Request with groovy spock tests outline expectations would be a great way to report such things in the future.

  2. bbakerman commented on Dec 17, 2017

    @bbakerman
    Member

    Looking further into this it is indeed the Scalar.XXX.parseValue method that is the source of the action.

    The graphql spec says

    http://facebook.github.io/graphql/October2016/#sec-String

    String
    Input Coercion
    When expected as an input type, only valid UTF‐8 string input values are accepted. All other input values must raise a query error indicating an incorrect type.

    So reading that the input object to a Scalar.GraphqlString should only be a valid UTF-8 string.

    However the graphql-js (reference implementation) has the following as its implementation

     function coerceString(value: mixed): ?string {
       if (Array.isArray(value)) {
         throw new TypeError(
            `String cannot represent an array value: [${String(value)}]`,
          );
      }
      return String(value);
    }
    

    So it will take any object value (other than a list) and use JavaScript co-ercion to make it a String. so number input, boolean input and so on.

    The graphql-java is also currently very liberal

    return input.toString();
    

    So the dilema we now have is that the spec says one thing but the reference implementation does another.

    So out there in graphql production land, a number variable input to a String scalar type will succeed but the spec says it shouldnt. This expectation versus specification is what I am struggling with here.

  3. Mark-K commented on Dec 18, 2017

    @Mark-K
    Author

    Thank you for looking into this very difficult and potentially contentious issue.

    I will look into how to do the pull request and making groovy spock tests for reporting future issues. Thank you for the suggestion.

    My thinking is that when the input data is JSON, the Map<String, Object> received as Input should have already been parsed by the JSON library's (such as Jackson) string to Map<String, Object> parser and converted into the Java type as specified in the input data and JSON value conversion rules. So in the case of correct value syntax, there should be no need for value coercion.

    For example:
    JSON:
    { "s": "s", "b": true, "n": 123 }
    should be parsed into
    Java Map<String, Object>
    { "s": String("s"), "b": Boolean.TRUE, "n": Integer(123) }

    As a result, the graphql-java library could require that the values of built-in scalars in the Input Map be the type specified by the schema, since the built-in scalars are basically the same as those defined by the JSON format. Though there may need to be some leniency with numeric type conversions such as up casting (Int -> Long) when necessary.

    This would mean that the code that prepares the variable map must do "the right thing".
    But that might not be current practice or not always reasonable, so making "strict" a requirement could break existing code and/or make parsing other data formats much more difficult. (i.e. web form data, Properties...)

    My suggestion, would be (if the code will be modified) if it is feasible to make strict and lenient input validation selectable (with the default being lenient perhaps so as not to break anyone's hard work). When explicitly set to "strict", values in the input map must be the type specified by the schema without coercion. The "lenient" selection would provide the current (best effort to make sense out of the input) behavior.

    This could be implemented in any of several different ways, for example:

    • an added boolean flag that changes the behavior of the existing validator
    • making input validation modular in the sense that a developer could select between a "StrictInputValidator" module, the "DefaultInputValidator" module or even implement a "UserDefinedInputValidator"
    • define pluggable "pre-validation" that executes just before field validation occurs
  4. mikeparisstuff commented on Dec 19, 2017

    @mikeparisstuff

    I believe I've stumbled upon this issue as well and as an aside would also love to see the validation system become more pluggable in the future.

    Given how the spec (http://facebook.github.io/graphql/October2016/#sec-String-Value) defines a string value I would think the implementation should validate that things that are being used as strings are indeed strings. I would expect the query in the snippet below to fail at validation time because I am trying to use a variable of type ID! for an argument of type String!.

    Curious what paths forward there are to fix this.

    Here is a snippet that isolates the issue.

    import graphql.ExecutionResult;
    import graphql.GraphQL;
    import graphql.schema.GraphQLSchema;
    import graphql.schema.StaticDataFetcher;
    import graphql.schema.idl.RuntimeWiring;
    import graphql.schema.idl.SchemaGenerator;
    import graphql.schema.idl.SchemaParser;
    import graphql.schema.idl.TypeDefinitionRegistry;
    
    import java.util.HashMap;
    
    import static graphql.schema.idl.RuntimeWiring.newRuntimeWiring;
    
    public class HelloWorld {
    
        public static void main(String[] args) {
            String schema = "schema { query: Query } \n" + "type Query{ hello(str: String!): String }";
    
            SchemaParser schemaParser = new SchemaParser();
            TypeDefinitionRegistry typeDefinitionRegistry = schemaParser.parse(schema);
    
            RuntimeWiring runtimeWiring = newRuntimeWiring()
                    .type("Query", builder -> builder.dataFetcher("hello", new StaticDataFetcher("world")))
                    .build();
    
            SchemaGenerator schemaGenerator = new SchemaGenerator();
            GraphQLSchema graphQLSchema = schemaGenerator.makeExecutableSchema(typeDefinitionRegistry, runtimeWiring);
    
            GraphQL build = GraphQL.newGraphQL(graphQLSchema).build();
            ExecutionResult executionResult = build.execute(
                    "query Get($str: ID!) { hello(str: $str) }",
                    "Get",
                    null,
                    new HashMap<String, Object>() {{ put("str", 1); }}
            );
            // This throws the error: Exception in thread "main" graphql.AssertException: wrappedType can't be null
        }
    }
  5. Mark-K commented on Dec 20, 2017

    @Mark-K
    Author

    Michael,
    Thanks for weighing in on this issue. You are reporting on a slightly different area than what I originally reported. My issue is with how the values of input variables are parsed (actually coerced) and validated. You are having a problem with how the query string itself is validated. This might deserve its own independent issue. But, having said that I have also seen (been surprised by) this behavior too.

    Oops! Sent type X in the query when type Y was required. Hahaha...

    Instead of an exception, an error with a more meaningful message would allow a client developer to understand/find/fix the mistake faster.

    Input variable definition parsing needs to take the following issues into account (at a minimum).

    • variable name mismatch: query ( $star: String! ) { hello ( str: $str ) }
    • variable Type mismatch:
      • query ( $str: String ) { hello( str: $str ) }
      • query ( $str: OtherThing ) { hello(str: $str ) }
      • query ( $str: OhterThing! ) { hello(str: $str ) }

    In most cases the library responds correctly; but, in some cases an exception gets thrown.
    The AssertionException is troublesome because it propagates all the way back to the application. If not handled by the application, no response is produced and the application unexpectedly terminates.
    When the application is responding to client input, unexpected termination is not a very polite way to say you messed up.

    Example 1 [OK]:
    The parameter name is misspelled in the query input variable definition. An error message describing the problem is returned to the client. Basically variable str wasn't found but did find an unused variable star in the input variables.
    Request: query Get($star: String!) { hello(str: $str) }
    Output: {"errors":[{"message":"Validation error of type UndefinedVariable: Undefined variable str","locations":[{"line":1,"column":40}]},{"message":"Validation error of type UnusedVariable: Unused variable star","locations":[{"line":1,"column":11}]}]}

    Example2 [OK]:
    The query is okay but the variable name in the variables map is misspelled. An error message is returned which when translated says that variable str was not found in the variables map.
    Request: query Get($str: String!) { hello(str: $str) }
    Variables: new HashMap<String, Object>() {{ put("star", 1); }}
    Output: {"errors":[{"message":"Variable 'str' has coerced Null value for NonNull type 'String!'"}]}

    Example3 [OK]:
    The input variable definition makes the str variable optional where as the schema says that its required. In this case as well, an error is returned to the client.
    Request: query Get($str: String) { hello(str: $str) }
    Output: {"errors":[{"message":"Validation error of type VariableTypeMismatch: Variable type doesn't match","locations":[{"line":1,"column":38}]}]}

    In most cases the graphql library's response is correct.

    For Schema:
    schema { query: Query }
    type Query { hello ( str: String!) : String }
    input MyInput { str: String ! }

    However, the library response is different depending on the input query string when the input variable type definition differs from the type specified by the schema. Testing results below show that Boolean returns an error message. But, 'Int!' throws an exception and 'Int' returns an error message with graphql claiming that 'Int' is an unknown type. Looking at the results below, when the input variable is a required String, an assertion exception will occur when other built-in scalar types Int, Long, ID (types other than Boolean) are specified.
    I believe that I have also experienced the assertion exception when the required input type is user defined and a different user defined input type was specified in the input variable definition but have no evidence and didn't reproduce that case this time so maybe I am wrong and it was actually an error message instead.

    Test Results:
    Query: query Get( $str : String ) { hello (str: $str) }
    Response: {"errors":[{"message":"Validation error of type VariableTypeMismatch: Variable type doesn't match","locations":[{"line":1,"column":38}]}]}

    Query: query Get( $str : Boolean! ) { hello (str: $str) }
    Response: {"errors":[{"message":"Validation error of type VariableTypeMismatch: Variable type doesn't match","locations":[{"line":1,"column":40}]}]}

    Query: query Get($str: Boolean) { hello(str: $str) }
    Response: {"errors":[{"message":"Validation error of type VariableTypeMismatch: Variable type doesn't match","locations":[{"line":1,"column":39}]}]}

    Query: query Get( $str : Int! ) { hello (str: $str) }
    Response: graphql.AssertException: wrappedType can't be null

    Query: query Get($str: Int) { hello(str: $str) }
    Response: {"errors":[{"message":"Validation error of type UnknownType: Unknown type Int","locations":[{"line":1,"column":17}]}]}

    Query: query Get($str: MyInput) { hello(str: $str) }
    Response: {"errors":[{"message":"Validation error of type VariableTypeMismatch: Variable type doesn't match","locations":[{"line":1,"column":39}]}]}

    Query: query Get($str: MyInput!) { hello(str: $str) }
    Response: {"errors":[{"message":"Validation error of type VariableTypeMismatch: Variable type doesn't match","locations":[{"line":1,"column":40}]}]}

    Here is a copy of the stack trace if that is helpful. It was produced by graphql-java-6.0.jar

    	at graphql.Assert.assertNotNull(Assert.java:10)
    	at graphql.schema.GraphQLNonNull.<init>(GraphQLNonNull.java:34)
    	at graphql.execution.TypeFromAST.getTypeFromAST(TypeFromAST.java:22)
    	at graphql.validation.TraversalContext.enterImpl(TraversalContext.java:141)
    	at graphql.validation.TraversalContext.enter(TraversalContext.java:78)
    	at graphql.validation.RulesVisitor.enter(RulesVisitor.java:58)
    	at graphql.validation.LanguageTraversal.traverseImpl(LanguageTraversal.java:33)
    	at graphql.validation.LanguageTraversal.traverseImpl(LanguageTraversal.java:36)
    	at graphql.validation.LanguageTraversal.traverseImpl(LanguageTraversal.java:36)
    	at graphql.validation.LanguageTraversal.traverse(LanguageTraversal.java:28)
    	at graphql.validation.Validator.validateDocument(Validator.java:40)
    	at graphql.GraphQL.validate(GraphQL.java:531)
    	at graphql.GraphQL.parseAndValidate(GraphQL.java:501)
    	at graphql.GraphQL.lambda$parseValidateAndExecute$2(GraphQL.java:482)
    	at graphql.execution.preparsed.NoOpPreparsedDocumentProvider.get(NoOpPreparsedDocumentProvider.java:11)
    	at graphql.GraphQL.parseValidateAndExecute(GraphQL.java:482)
    	at graphql.GraphQL.executeAsync(GraphQL.java:463)
    	at graphql.GraphQL.execute(GraphQL.java:394)
    	at graphql.GraphQL.execute(GraphQL.java:353)
    
  6. andimarek commented on Jul 19, 2019

    @andimarek
    Member

    closing this fairly old issue now . Please open a new issue against the current graphql java version if there is still a problem with input coercion.
    I am sorry that we didn't address this topic in time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions