PostgreSQL: Optimizing Aggregates

Aggregation of vast energy: our sun.

It's time to write about PostgreSQL 9.6 again. With the Release Candidate 1 out in the wild, we slowly approach the end of a very interesting development cycle. Here I would like to talk about the development in the field of aggregates. So, two commits, 9/552 and 9/435, will improve the performance of queries using aggregate functions and GROUP BY clauses:

PostgreSQL: Index-Only Scans with Partial Indexes

Partial sun lurking out of the water.

Another posts of my PostgreSQL 9.6 series. This time, I am talking about commit 9/299.  The complete discussion can be found here.

PostgreSQL: Parallel Aggregate

With PostgreSQL 9.6 looming on the horizon, I went out to sift through some of PostgreSQL's commitfests to find some interesting bits and pieces. This post is the start of a series covering commits of the next generation of the venerable database management system.

Outflow made parallel.

Even Faster Heaps

An ambulance rushing by.

Heaps are about performance. So, it is time to make xheap faster again. After realizing that the actual slowdown of RemovalHeap and XHeap does not simply stem from the general overhead but from NOT using the C implementation at all, I decided to change that.

Designing xfork

Recently, I came to know a small team working on a problem which they try to solve by using threads. As expected, problems popped up soon and development slowed down considerably. So based on the previous post, I would like to lay out my intentions and design decisions regarding xfork, a module I've written and actively maintain in analysis of the newly introduced async/await syntax.

Concurrency is a hard engineering problem.

Take it seriously and even consider not being concurrent a valid option.

Design Assumptions

I created xfork from the following observations based on my own experience. Developers usually:

Heaps in Python

This time, we dig into the matter of how we can use heaps in Python. As a starter, you want heaps when there's a job to do like the following:

queue=[]
while necessary:
    ...
    queue.append(an_item)   # or two
    ...
    ...
    next_item = min(heap)   # what's next
    ...
    queue.remove(next_item) # we're done; remove it
    ...