Skip to content

Enumerator.peek not working with Enumerators created with #to_enum over ActiveRecord Relations #6309

@dgolombek

Description

@dgolombek

Environment Information

  • JRuby version: jruby 9.2.11.1 (2.5.7) 2020-03-25 b1f55b1 OpenJDK 64-Bit Server VM 25.232-b09 on 1.8.0_232-b09 +jit [darwin-x86_64]
  • OS: Darwin MBP.local 18.7.0 Darwin Kernel Version 18.7.0: Mon Feb 10 21:08:45 PST 2020; root:xnu-4903.278.28~1/RELEASE_X86_64 x86_64
  • ActiveRecord *: 4.2.11.3

Expected Behavior

When I create an Enumerator over an ActiveRecord Relation using #to_enum, #peek on that enumerator does not always work. The exact behavior seems to vary a bit -- in my sample test, #peek throws StopIteration, but when I run this in larger environments, the Enumerator seems to hang. This may be tied to to AR Model size (and thus load time) -- I have a harder time reproducing with my small models, and easier with big, bloated objects.

https://gist.github.com/dgolombek/3123d5602ff6d76e96f2bac01b570210

Actual Behavior

Failures:

  1. Enumerator with a single value in an AR array should not hang
    Failure/Error: raise "bad first element on first peek, #{iter.peek.id} != #{first.id}" unless iter.peek.id == first.id

    StopIteration:
    iteration reached an end

    uri:classloader:/jruby/kernel/enumerator.rb:25:in `next'

    uri:classloader:/jruby/kernel/enumerator.rb:34:in `peek'

    ./spec/simple_enum_spec.rb:4:in `check_iter'

    ./spec/simple_enum_spec.rb:25:in `block in '

    ./spec/spec_helper.rb:113:in `block in '

    ./spec/spec_helper.rb:112:in `block in '

    ./spec/spec_helper.rb:111:in `block in '

    ./spec/spec_helper.rb:100:in `block in '

    ./spec/spec_helper.rb:99:in `block in '

Using MRI 2.6.6, this works correctly. As mentioned above, when I deployed to prod, we saw uncontrolled thread growth, leading to server death, which I believe was tied to threads hanging in the enumerator (but could be tied to bugs with our handling of unexpected StopIteration, I'm still debugging that). I haven't gotten thread dumps yet, will work on doing so.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions