Skip to content

Possible performance improvement for heapq.merge() #83119

Closed

Description

@sweeneyde
BPO 38938
Nosy @tim-one, @rhettinger, @serhiy-storchaka, @bharel, @DimitrisJim, @bbayles, @sweeneyde
PRs
  • bpo-38938: Added special case to heapq.merge for small numbers of iterables #17422
  • bpo-38938: Adjusting the heapq.merge algorithm for fewer comparisons #17729
  • bpo-38938: Reduce the number of comparisons in heapq.merge #18427
  • bpo-38938: Optimize heapq.merge() #20550
  • Files
  • new_merge.py: Iterative version for comparison
  • tournament_heap.py: Using a heap that stores each item only once, items move from leaves to root.
  • recursive_merge.py: Recursive 2-way-merging generators
  • losers.py: Using a "loser tree"
  • winners.py: like new_merge, except jump straight to leaf instead of traversing down to it
  • key_and_reverse.py: Similar to winners.py but handles key and reverse; also uses lists as node structs.
  • new_merge2.py: Tree formation is iterative and inlined
  • less_movement.py
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/rhettinger'
    closed_at = None
    created_at = <Date 2019-11-29.02:44:20.990>
    labels = ['type-feature', 'library', '3.10']
    title = 'Possible performance improvement for heapq.merge()'
    updated_at = <Date 2022-01-19.20:29:35.667>
    user = 'https://github.com/sweeneyde'

    bugs.python.org fields:

    activity = <Date 2022-01-19.20:29:35.667>
    actor = 'bar.harel'
    assignee = 'rhettinger'
    closed = False
    closed_date = None
    closer = None
    components = ['Library (Lib)']
    creation = <Date 2019-11-29.02:44:20.990>
    creator = 'Dennis Sweeney'
    dependencies = []
    files = ['49160', '49164', '49166', '49167', '49170', '49180', '49184', '49198']
    hgrepos = []
    issue_num = 38938
    keywords = ['patch']
    message_count = 23.0
    messages = ['357631', '357632', '357664', '357665', '358968', '364160', '364198', '364201', '364205', '364260', '364279', '364297', '368934', '368989', '369012', '369064', '369115', '369145', '369198', '369559', '369667', '370176', '370413']
    nosy_count = 7.0
    nosy_names = ['tim.peters', 'rhettinger', 'serhiy.storchaka', 'bar.harel', 'Jim Fasarakis-Hilliard', 'bbayles', 'Dennis Sweeney']
    pr_nums = ['17422', '17729', '18427', '20550']
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue38938'
    versions = ['Python 3.10']

    Activity

    1. sweeneyde commented on Nov 29, 2019

      @sweeneyde
      MemberAuthor

      Although the implementation of the heapq.merge function uses an underlying heap structure, its behavior centers on iterators. For this reason, I believe there should either be an alias to this function in the itertools module or at least a recipe in the itertools docs describing the use of heapq.merge.

      Furthermore, I have an implementation (attached) of heapq.merge that is twice as fast for two iterables (as used in mergesort and other common cases), and is faster for up to 6 or 7 iterables. I think it would be nice to special-case this for small numbers of iterables for this significant speedup.

    2. added
      stdlibStandard Library Python modules in the Lib/ directory
      type-featureA feature request or enhancement
      on Nov 29, 2019
    3. sweeneyde commented on Nov 29, 2019

      @sweeneyde
      MemberAuthor

      The following seems like it is a short, readable recipe for itertools.

    4. self-assigned this
      on Nov 30, 2019
    5. sweeneyde commented on Nov 30, 2019

      @sweeneyde
      MemberAuthor

      Disregard merge_recipe.py: it would skip over a value that had already been retrieved from the iterator when the loop finished.

    6. rhettinger commented on Nov 30, 2019

      @rhettinger
      Contributor

      Several thoughts:

      • heapq.merge() could have plausibly been in the itertools module instead of the heapq module. However, since it already has a home, there is no reason to move it or to create duplication with an intermodule dependency. We certainly don't have a rule that the itertools module must house everything accepts an iterator as an input and returns an iterator in the output.

      • The iter_merge() recipe is a bit sprawling and IMHO not a useful teaching aid (a primary goal of the itertools recipe), so it doesn't have much documentation value. Instead, consider submitting this to the "more-itertools" project which is referenced by the docs. There it would likely be used directly rather than as a recipe.

      • Over the next few days, I'll take a closer look at the binary tree approach versus a heap approach. I like how it intrinsically avoids the overhead of the 4-tuple while maintaining order, but I want to test how well it does with slow comparison keys (i.e. does the total number of comparisons get worse). Am not sure whether the tree needs to be rebalanced as some of the input streams become exhausted. I have a vague recollection that Knuth's TAOCP deemed the heap approach to be algorithmically superior, but need to dig out my books to be verify that.

      • We could just build-out the current heapq.merge() code to include a special case for only two input iterables. That would speed-up a common case by avoiding the overhead of tracking a 4-tuple for each element. If you want to submit a PR for that, I would be happy to take a close look and verify the timings.

      • It might be interesting to time the pure python variants against "sorted(chain(a, b, c, d, e))". It isn't quite an apples-to-apples comparison because the latter pulls all the data in memory and it doesn't the runs pre-segregated, but it might give a sense of how well merge() could do if we decided to go gonzo on optimizations. Until now, we've been satisfied with letting the heap structure minimize the number of comparisons and there hasn't been a focus on reducing the constant factor overhead.

    7. changed the title [-]Iterable merge performance and inclusion in itertools[/-] [+]Possible performance improvement for heaqq.merge()[/+] on Nov 30, 2019
    8. sweeneyde commented on Dec 28, 2019

      @sweeneyde
      MemberAuthor

      PR 17729 is a C implementation of a non-recursive "flattening" of the the recursive-lazy-mergesort algorithm into a tournament whose state is a tree of losers of comparisons.

    9. serhiy-storchaka commented on Mar 14, 2020

      @serhiy-storchaka
      Member

      This is a large change. And making heaqq.merge a class looks unrelated to the declared goal. I afraid it may add significant overhead.

      Since this is an optimization, could you please provide any benchmarks which we can use to check the performance boost?

    10. 7 remaining items

    11. rhettinger commented on May 16, 2020

      @rhettinger
      Contributor

      The nested generators approach looks promising. I hope you keep working on that :-)

      I'm not too keen on PR 18427 because it is a massive explosion in code volume and complexity.

      With some continued effort, I expect we'll get to something much simpler and more maintainable. The existing code already performs well for typical applications, so there is no need to "go gonzo" and throw a massive wall of code at the problem. The ensuing maintenance costs would be too high.

      To save time and effort, please hold-off on a C implementation until we've agreed to a pure python version of the algorithm.

      For timings, try using strings, tuples, or dataclass instances. Timing integer inputs isn't representative of how merge() is used and it tends to exaggerate improvements. For interesting merges, the dominant cost is the comparison step.

      By switching to a binary tree arrangement, the payoff we're aiming for is to avoid the double comparison in the tuple inequality logic — "('a',1)<('b',2)" tests both 'a'=='b' and 'a'<='b'. I suggest aiming for the simplest possible code that avoids the double test.

    12. sweeneyde commented on May 16, 2020

      @sweeneyde
      MemberAuthor

      The attached recursive_merge.py should be much less ugly and still somewhat performant.

      It should be the same algorithm as that PR, just written recursively rather than iteratively.

      I got some text files from http://www.gwicks.net/dictionaries.htm and tried merging them line-by-line:

      py -3.9 -m pyperf timeit -s "from heapq import merge; from collections import deque" "deque(merge(open('english.txt'), open('dutch.txt'), open('french.txt'), open('german.txt'), open('italian.txt')), maxlen=0)"

      Mean +- std dev: 391 ms +- 9 ms
      

      py -3.9 -m pyperf timeit -s "from recursive_merge import merge; from collections import deque" "deque(merge(open('english.txt'), open('dutch.txt'), open('french.txt'), open('german.txt'), open('italian.txt')), maxlen=0)"

      Mean +- std dev: 262 ms +- 9 ms
      

      Perhaps that's a more real-world benchmark.

    13. rhettinger commented on May 16, 2020

      @rhettinger
      Contributor

      For comparison, I'm attaching an iterative version that manipulates the tree nodes with all local variables. Try it out and see what you think.

    14. sweeneyde commented on May 17, 2020

      @sweeneyde
      MemberAuthor

      It seems to me that the code sprawl mostly comes from the separate handling of the four keyed/unkeyed and forward/reverse cases, which as far as I can tell requires a branch in the innermost loop if not unrolled into separate cases. I think recursive_merge.py is about as concise as possible while still efficiently handling all four cases.

      Is there any issue with recursion in this application? Even if there are 2**64-1 iterables, this would only mean 64 nested next() calls, which should be within the stack on most machines, no?

      I did the following benchmark:

      py -3.9 -m pyperf timeit -s "from [FILENAME] import merge; from collections import deque" "deque(merge(open('english.txt'), open('dutch.txt'), open('french.txt'), open('german.txt'), open('italian.txt')), maxlen=0)"

      new_merge.py:          Mean +- std dev: 773 ms +- 16 ms
      tournament_heap.py:    Mean +- std dev: 665 ms +- 42 ms
      losers.py:             Mean +- std dev: 470 ms +- 31 ms
      Existing heapq:        Mean +- std dev: 313 ms +- 2 ms
      recursive_merge.py:    Mean +- std dev: 266 ms +- 3 ms
      

      I can look at some more diverse benchmarks (types, iterable length imbalance, lengths of an iterator's win-streak, etc.) soon.

    15. rhettinger commented on May 17, 2020

      @rhettinger
      Contributor

      Am leaning toward the iterative approach in new_merge.py because it most directly implements the core concept of storing the data in a binary tree.

      Merits: no recursion, no nested generators, avoids the clutter of left-empty and right-empty checks, easy to understand, simple and fast loops, all local variables or instance variables, no calls to memory allocator after then initial tree construction, runs well on PyPy, and easily Cythonized.

    16. sweeneyde commented on May 18, 2020

      @sweeneyde
      MemberAuthor

      I mostly like new_merge.py too, especially the dynamic reduction of the tree.

      However, it looks like list(merge([2],[1],[1])) currently fails, and I think what's missing is the following in the sibling-promotion:

      + if sibling.left is not None:
      + sibling.left.parent = sibling.right.parent = parent

      Also for what it's worth, I think both appearances of "c1 if c1.value < c2.value else c2" should become "c2 if c2.value < c1.value else c1" for stability.

      I've attached winners.py, which is similar to new_merge.py, but replaces the "while node.left is not None: node = node.source" traversal with a single "node = node.leaf" call and adds a fast yield-from for the last iterator. I don't think it's much more complex either.

    17. sweeneyde commented on May 22, 2020

      @sweeneyde
      MemberAuthor

      key_and_reverse.py employs the same strategy as winners.py, but uses lists as the nodes of the tree rather than using Node instances. It also eliminates the recursion of treeify, and adds (with neither much of a performance hit nor much code duplication) support for key=some_func and/or reverse=True keyword arguments, so it now passes the test_heapq suite.

    18. rhettinger commented on May 22, 2020

      @rhettinger
      Contributor

      Thanks. I'll look at this shortly. We're getting much closer.

      Just so that I don't lose other work I've done, am uploading new_merge2.py with in-line iterative tree construction of the initial tree.

    19. sweeneyde commented on May 28, 2020

      @sweeneyde
      MemberAuthor

      less_movement.py is my favorite so far. It still handles key and reverse,
      but using instance attributes instead of the list indices I tried before.
      It does this by only advancing the "key" and "leaf" attributes up toward
      the root (where the key is just the value if key is None), while the value
      is stored only in the leaf. Since the "value" attribute is no longer used
      except for at the leaves, we can pack a leaf's value into its left slot
      and reduce the number of slots.

      This seems to run very well on pypy and it looks like it would be pretty
      receptive to a c implementation, which I would be happy to work on.

    20. changed the title [-]Possible performance improvement for heaqq.merge()[/-] [+]Possible performance improvement for heapq.merge()[/+] on May 28, 2020
    21. sweeneyde commented on May 31, 2020

      @sweeneyde
      MemberAuthor

      PR 20550 uses a linked structure like what we've been talking about.

    22. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    3.10 (EOL)end of lifestdlibStandard Library Python modules in the Lib/ directorytype-featureA feature request or enhancement

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions