Skip to content

DataLoader support for Mutations #3711

Closed

Description

@rstoyanchev

We've had a report about batch loading not working with mutations, which seemed counter intuitive since query and mutation should work the same way for field selection. On closer look, however, DataLoader does expect AsyncExecutionStrategy while mutations execute with AsyncSerialExecutionStrategy. That's probably related to the requirement for serial execution of mutations, but still, batch loading not working seems more like an unintended side effect. It would be good to make this work if it can, or otherwise clarify the limitations, and update the DataLoader documentation accordingly.

Activity

  1. dondonz commented on Oct 29, 2024

    @dondonz
    Member

    Hello, sorry for the late reply, team has been away on holidays. I'll get back to you!

  2. bbakerman commented on Nov 10, 2024

    @bbakerman
    Member

    I started to look into this to see why its not working.

    The first reason is that we have code that says "no efficient dataloader dispatching" on mutations in the code bvase. So this is the primary reason this is not working.

    However POC removed the "restriction" on "non mutations" like we do on query (so this is slightly different to master but otherwise vert close) but several tests hang and never complete.

    This is what I think its happening - I am not 100% sure yet but I have some evidence towards this conclusion

    In a normal (query) execution AsyncExecutionStrategy#execute is called and dataLoaderDispatcherStrategy.executionStrategy(executionContext, parameters); is called.

    This sets up an expectation that N top fields are expected to be dispatched. And once N have been dispatched, the DataLoader dispatching can happen.

    However mutations use AsyncSerialExecutionStrategy#execute - it too calls this dataLoaderDispatcherStrategy.executionStrategy(executionContext, parameters); and hence
    if there are N mutation fields, it will only call dispatch when N fields are dispatched

    HOWEVER

    mutations run in field order - eg for 3 top level mutation fields,

    mutation m {
        m1
        m2
        m3
    }
    

    we must do field m1 first and when that completes, we must do field m2 and so on to m3`

    However the current dataLoaderDispatcherStrategy.executionStrategy(executionContext, parameters); code sets up an exception for N field (in my example above 3) before it dispatches but there will never be 3 fields all dispatched. There will only ever be 1 top level field at a time

    I think we need a new method that sets up an expectation of 1 field for "serial" async processing. Maybe some new dataLoaderDispatcherStrategy.executionSerialStrategy(executionContext, parameters); that sets up an expectation of 1 field and then when that fields dispatched we, we loop again to the next field and setup a new expectation etc...

    I haven't tried this yet but I can see that it locks up (does not DataLoader dispatching) because it setup a expectation of N top level fields but only completed 1.

  3. added
    keep-openTells Stale Bot to keep PRs and issues open
    on Nov 14, 2024
  4. removed
    keep-openTells Stale Bot to keep PRs and issues open
    on Dec 1, 2024
  5. github-actions commented on Jan 31, 2025

    @github-actions
    Contributor

    Hello, this issue has been inactive for 60 days, so we're marking it as stale. If you would like to continue this discussion, please comment within the next 30 days or we'll close the issue.

  6. andimarek commented on Jan 31, 2025

    @andimarek
    Member

    This should be merged cc @bbakerman

  7. palhoye commented on Jan 31, 2025

    @palhoye

    Hello, this issue has been inactive for 60 days, so we're marking it as stale. If you would like to continue this discussion, please comment within the next 30 days or we'll close the issue.

    Just want to comment to encourage this issue to be prioritized and resolved. Since most people are not aware it causes performance problems, and once aware the workaround is to perform a separate query after the mutation completes. So I feel it's an important issue to resolve and it should not be closed.

  8. dondonz commented on Feb 1, 2025

    @dondonz
    Member

    Hello I had a look at the related branch and it already had been merged in at the end of November. It will be included in the next release

    I'll double check in case there's any other work planned for this issue. Taking the "stale" tag off for now

  9. added
    keep-openTells Stale Bot to keep PRs and issues open
    on Feb 1, 2025
  10. added this to the 23.x breaking changes milestone on Feb 1, 2025
  11. dondonz commented on Feb 6, 2025

    @dondonz
    Member

    Confirming the PR includes all the fixes, this issue is done #3737

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    keep-openTells Stale Bot to keep PRs and issues open

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions