Merging dictionaries in Python

How can you merge together dictionaries in Python?

It depends on what you mean by "merge together".

Merging dictionaries with | in Python

First let's talk about the simplest way to merge dictionaries, which will usually be all that you need.

Here are two dictionaries:

>>> context = {"language": "en", "timezone": "UTC"}
>>> more_context = {"title": "Home", "breadcrumbs": ["Home"]}

We'd like to make a new third dictionary that combines these two. This new dictionary should contain all the key-value pairs from both dictionaries.

The easiest way to do this is to use the pipe (|) operator:

>>> new_context = context | more_context

This made a new dictionary that contains all the items from both of the two initial dictionaries:

>>> new_context
{'language': 'en', 'timezone': 'UTC', 'title': 'Home', 'breadcrumbs': ['Home']}

Using the | operator is basically the same as making a new empty dictionary, and then looping over all the items in the first dictionary and then all the items in the second dictionary, and adding all of them to the new dictionary:

>>> new_context = {}
>>> for key, value in context.items():
...     new_context[key] = value
...
>>> for key, value in more_context.items():
...     new_context[key] = value
...
>>> new_context
{'language': 'en', 'timezone': 'UTC', 'title': 'Home', 'breadcrumbs': ['Home']}

Merging dictionaries with the update method

What if we wanted to update a dictionary in-place?

Let's say we have the same two dictionaries as before:

>>> context = {"language": "en", "timezone": "UTC"}
>>> more_context = {"title": "Home", "breadcrumbs": ["Home"]}

And we want to update our first dictionary (context) to also contain all the items from the second dictionary (more_context).

We could loop over our second dictionary and add each key-value pair to the first dictionary:

>>> for key, value in more_context.items():
...     context[key] = value
...
>>> context
{'language': 'en', 'timezone': 'UTC', 'title': 'Home', 'breadcrumbs': ['Home']}

But the dictionary update method can do that work for us:

>>> context.update(more_context)

The update method accepts a dictionary and modifies the dictionary that it's called on to contain all the key-value pairs from both dictionaries.

It's also possible to use the | operator in an augmented assignment statement:

>>> context |= more_context

This does the same thing as the dictionary update method, so which one you use is up to your personal preference.

Merging dictionaries with **

We can also merge dictionaries with Python's double star (**) syntax:

>>> combined = {**context, **more_context}

This creates a new dictionary, just like the pipe operator (|).

I find {**a, **b} a little bit less readable than a | b, so I usually use the | operator instead of ** to merge dictionaries.

The difference between | and **

It's worth noting that the | operator and the ** syntax don't do quite the same thing.

Sometimes a | b and {**a, **b} will provide a different result.

For example, if you use the pipe operator with a custom mapping type, like defaultdict, that custom type will be used for the new dictionary.

Here are two defaultdict objects:

>>> from collections import defaultdict
>>> group1 = defaultdict(list)
>>> group1["giraffe"].append("Gerard")
>>> group2 = defaultdict(list)
>>> group2["ferret"].append("Francis")
>>> group1
defaultdict(<class 'list'>, {'giraffe': ['Gerard']})
>>> group2
defaultdict(<class 'list'>, {'ferret': ['Francis']})

If we use the pipe operator to merge these two defaultdict objects together, we'll end up getting a defaultdict object in return:

>>> group1 | group2
defaultdict(<class 'list'>, {'giraffe': ['Gerard'], 'ferret': ['Francis']})

If we use ** to merge these two dictionaries, we'll get a dict object instead:

>>> {**group1, **group2}
{'giraffe': ['Gerard'], 'ferret': ['Francis']}

The type of the first dictionary is used whenever the pipe operator (|) is used to merge two dictionaries. But when ** is used to merge dictionaries, we'll always get a dict object.

So occasionally I'll use ** if I want to accept any type of dictionary, but I'd like to always return the built-in dict type. But usually, the behavior of | is actually what I'm looking for.

Handling duplicate keys when merging

What would happen if we merge two dictionaries that have overlapping keys?

For example, our two dictionaries here both have a title key:

>>> context = {"language": "en", "timezone": "UTC", "title": "Welcome"}
>>> more_context = {"title": "Home", "breadcrumbs": ["Home"]}

Well, just as if we merged our dictionaries by looping over them with for loops, it's the last value for each key that will end up in our dictionary.

Since more_context is after context here, the value for title will end up being Home (instead of Welcome).

>>> context | more_context
{'language': 'en', 'timezone': 'UTC', 'title': 'Home', 'breadcrumbs': ['Home']}

What if we want a different behavior?

For example, what if we wanted to handle duplicate keys by making the largest value be the one that appears in our new dictionary? When merging these two dictionaries of prices we would want the merged dictionary to contain the largest amount whenever keys overlap:

>>> prices1 = {"premium": 29.99, "basic": 9.99, "pro": 49.99}
>>> prices2 = {"basic": 7.99, "pro": 39.99}

Unfortunately, there's no simple shortcut for this case.

We can do this by copying the first dictionary, looping over the second one, checking whether each key is in our new dictionary while comparing its value (using get), and updating its value appropriately:

>>> merged = prices1.copy()
>>> for plan, price in prices2.items():
...     if price > merged.get(plan, 0):
...         merged[plan] = price
...
>>> merged
{'premium': 29.99, 'basic': 9.99, 'pro': 49.99}

The | operator performs a "union" between dictionaries

Usually, the easiest way to join two dictionaries together is to use the pipe operator.

>>> context = {"language": "en", "timezone": "UTC"}
>>> more_context = {"title": "Home", "breadcrumbs": ["Home"]}
>>> context | more_context
{'language': 'en', 'timezone': 'UTC', 'title': 'Home', 'breadcrumbs': ['Home']}

But why does Python use the pipe operator (|) for this operation instead of the plus operator (+)?

>>> context + more_context
Traceback (most recent call last):
  File "<python-input-4>", line 1, in <module>
    context + more_context
    ~~~~~~~~^~~~~~~~~~~~~~
TypeError: unsupported operand type(s) for +: 'dict' and 'dict'

The plus operator is used for joining sequences with concatenation. But concatenation is a little bit different from joining dictionaries.

Dictionaries need to handle the case of joining with duplicate keys, but sequences don't.

The pipe operator was chosen because it was already being used for joining together sets:

>>> names = {"language", "timezone", "title"}
>>> more_names = {"title", "breadcrumbs"}
>>> names | more_names
{'language', 'timezone', 'breadcrumbs', 'title'}

The pipe operator performs a "union" operation between sets.

You can think of merging dictionaries as similar to unioning two dictionaries together.

Join dictionaries with the | operator

The next time you need to join two dictionaries together, remember the pipe operator (|).

But if you need to customize the way you join dictionaries together, you'll probably need a for loop instead.