Comprehending Python’s Comprehensions

One of my favorite features in Python are list comprehensions. They can seem a bit arcane at first but when you break them down they are actually a very simple construct.

The key to understanding list comprehensions is that they’re just for-loops over a collection expressed in a more terse and compact syntax. Let’s take the following list comprehension as an example:

>>>squares=[x*xforxinrange(10)]

It computes a list of all integer square numbers from 0 to 9:

>>>squares[0,1,4,9,16,25,36,49,64,81]

If we wanted to build the same list using a plain for-loop we’d probably write something like this:

>>>squares=[]>>>forxinrange(10):...squares.append(x*x)

That’s a pretty straightforward loop, right? If you try and generalize some of this structure you might end up with a template similar to this:

(values)=[(expression)for(value)in(collection)]

The above list comprehension is equivalent to the following plain for-loop:

(values)=[]for(value)in(collection):(values).append((expression))

Again, a fairly simple cookiecutter pattern you can apply to most for loops. Now there’s one more useful element we need to add to this template, and that is element filtering with conditions.

List comprehensions can filter values based on some arbitrary condition that decides whether or not the resulting value becomes a part of the output list. Here’s an example:

>>>even_squares=[x*xforxinrange(10)ifx%2==0]

This list comprehension will compute a list of the squares of all even integers from 0 to 9.

If you’re not familiar with what the modulo (%) operator does—it returns the remainder after division of one number by another. In this example the %-operator gives us an easy way to test if a number is even by checking the remainder after we divide the number by 2.

>>>even_squares[0,4,16,36,64]

Similarly to the first example, this new list comprehension can be transformed into an equivalent for-loop:

even_squares=[]forxinrange(10):ifx%2==0:vals.append(x)

Let’s try and generalize the above list comprehension to for-loop transform again. This time we’re going to add a filter condition to our template to decide which values end up in the resulting list.

Here’s the list comprehension template:

values=[expressionforvalueincollectionifcondition]

And we can transform this list comprehension into a for-loop with the following pattern:

vals=[]forvalueincollection:ifcondition:vals.append(expression)

Again, this is a straightforward transformation—we simply apply our cookiecutter pattern again. I hope this dispelled some of the “magic” in how list comprehensions work. They’re really quite a useful tool.

Before you move on I want to point out that Python not only supports list comprehensions but also has similar syntax for sets and dictionaries.

Here’s what a set comprehension looks like:

>>>{x*xforxinrange(-9,10)}set([64,1,36,0,49,9,16,81,25,4])

And this is a dict comprehension:

>>>{x:x*xforxinrange(5)}{0:0,1:1,2:4,3:9,4:16}

Both are useful tools in practice. There’s one caveat to Python’s comprehensions—as you get more proficient at using them it becomes easier and easier to write code that’s difficult to read. If you’re not careful you might have to deal with monstrous list, set, dict comprehensions soon. Remember, too much of a good thing is usually a bad thing.

After much chagrin I’m personally drawing the line at one level of nesting for comprehensions. I found that in most cases it’s better (as in “more readable” and “easier to maintain”) to use for-loops beyond that point.