Comprehensions

In the previous lesson, you learned how to clean a string and turn it into a list of words. Very often, you want to build a new list from an existing one. You already know how to do that with a for loop. But Python gives you another way to write that kind of operation, and that is what this lesson explores.

Start with something familiar: the restaurant receipt from earlier lessons. It had three items, Pasta, Salad and Soup, with prices of 12.50, 8.25 and 6.00. Suppose you want to add eight percent tax to each price, and store the updated prices in a new list.

The same operation in one line

There is nothing wrong with this code. It is clear, and it works perfectly well. But Python lets you write the same operation in just one line. Take a look at this cell:

with_tax = [round(p * 1.08, 2) for p in prices]
with_tax

[13.5, 8.91, 6.48]

When you run it, you get exactly the same three values, in the same order.

So what is happening here? How can one expression do the same job as the entire loop? And more importantly, how do you read and write expressions like this without getting confused? This feature is called a list comprehensionKey termA list built from an expression evaluated for each item of an iterable: [round(p * 1.08, 2) for p in prices], with an optional if at the end to filter., and the next section breaks it down.

Inside the square brackets, the comprehension has two main parts:

[round(p * 1.08, 2) for p in prices]

[13.5, 8.91, 6.48]

The expression

The first part is the call to round: round(p * 1.08, 2). That is the expression Python evaluates for each item. It is exactly the same calculation you passed to append inside the loop.

The for clause

Then comes for p in prices. That is the loop. Notice that there is no colon at the end of it, the way there is in a for statement.

Finally, the square brackets around everything tell Python to build a new list. So you do not need to create an empty list yourself, and you do not need to call append. Python handles that for you.

That is the basic idea of a list comprehension: it builds a new list by evaluating an expression for each item in an iterable.

Tracing it

Here is what happens, one iteration at a time:

• On the first iteration, p is 12.50. Python calculates 12.50 * 1.08 and rounds it to two decimal places, then adds 13.5 to the new list.
• On the second iteration, p is 8.25, so the result is 8.91.
• On the third iteration, p is 6.00, so the result is 6.48.

By the end, Python has built a list containing all three results, in the same order as the original prices.

So far, you have created one output value for every input value. But not every loop you have written worked that way. Sometimes you only wanted to keep certain items.

In the for loops lesson, you worked with a week of daily step counts. Return to that example for a moment. Suppose you want to build a list containing only the step counts that are greater than or equal to ten thousand.

Using a regular loop, you would start with an empty list called active, go through each step count in week, and only append the value if it meets the condition:

week = [8200, 10450, 6300, 12000, 9100, 7400, 11050]
active = []
for s in week:
    if s >= 10000:
        active.append(s)
active

[10450, 12000, 11050]

The result contains three values. Those are the three days when you reached at least ten thousand steps.

A condition in a comprehension

Can you write this as a comprehension? Absolutely. You just need to add an if condition:

[s for s in week if s >= 10000]

[10450, 12000, 11050]

This list comprehension does exactly the same thing, and you get exactly the same three values.

Look carefully at where the if appears. It comes after the for clause. So you start with the expression you want to keep, then the loop, and finally the condition. This condition acts as a filterKey termA condition that determines which items are included..

Tracing the filter

Python goes through every step count, but it only includes a value in the result if the condition is True. If the condition is False, that value is skipped.

So the original week contains seven step counts, but the resulting list contains only three, because four of those days did not meet the condition.

That is an important distinction. A comprehension does not always produce the same number of elements as its input. When you add a filter, some elements can be left out.

There is another way to use conditions inside comprehensions, and this is where things can get a little confusing: the if appears in a different position, and it does a different job.

Go back to the weekly step counts. Suppose you do not want to remove any days. Instead, you want to label every day. If the step count is at least ten thousand, the label is "Active". Otherwise, the label is "Rest". So you want all seven labels, one for each day of the week.

Two kinds of condition

That is the key difference between these two kinds of condition. When you write a comprehension the way you did earlier:

[s for s in week if s >= 10000]

[10450, 12000, 11050]

the if at the end is a filter. It decides whether an item should be included at all: seven inputs, three included values.

But when you write it like this:

["Active" if s >= 10000 else "Rest" for s in week]

['Rest', 'Active', 'Rest', 'Active', 'Rest', 'Rest', 'Active']

the if and else are part of the expression at the beginning. They decide which value to produce for every item: seven inputs, seven labels.

So one approach filters elements, and the other transforms every element based on a condition.

Comprehensions are not limited to lists. You can also use them to create sets and dictionaries, and the syntax is very similar.

Set comprehensions

Start with sets. Remember the restaurant items, Pasta, Salad and Soup? Suppose you want to find the different lengths of those item names. You can use a set comprehensionKey termA set built from an expression evaluated for each item: {len(item) for item in items}.. It looks almost exactly like a list comprehension, but instead of square brackets, you use curly brackets:

items = ["Pasta", "Salad", "Soup"]
{len(item) for item in items}

{4, 5}

Here is what happens. Pasta has five letters. Salad also has five letters. And Soup has four letters. So the expression produces 5, 5 and 4. But because you are building a set, duplicate values are removed, and the result contains just two values: 4 and 5.

So a set comprehension works like a list comprehension, except that Python builds a set instead of a list.

Dictionary comprehensions

What about dictionaries? As you know, dictionaries store key-value pairs. So when you write a dictionary comprehensionKey termA dictionary built from a key and a value evaluated for each item: {item: price for item, price in zip(items, prices)}., you need to provide both a key and a value, separated by a colon.

Use the restaurant example again. You have a list of item names, items, and another list of prices, prices. Now bring back the zip function from the for loops lesson, which you used to go through two sequences together:

menu = {item: price for item, price in zip(items, prices)}
menu

{'Pasta': 12.5, 'Salad': 8.25, 'Soup': 6.0}

Here is what is happening. First, zip() produces pairs of item names and prices. On each iteration, Python unpacks the pair into item and price. Then the expression at the front, item: price, creates a key-value entry: the item name becomes the key, and the price becomes the value.

So the resulting dictionary contains Pasta with 12.5, Salad with 8.25, and Soup with 6.0. That is exactly the same menu you created in the dictionaries lesson.

The important question is: when should you use a comprehension in the first place, and when is a regular for loop better?

The main purpose of a comprehension is to build a collection: a list, a set or a dictionary. It is especially useful when you are transforming or filtering values.

But what if the loop's main job is to perform an action, for example printing something? Then a regular loop is the obvious choice:

for item in items:
    print(item)

Pasta
Salad
Soup

Stopping early

There is one more limitation. A comprehension always goes through every item. If you need to stop at a particular point, use a regular for loop with break:

for s in week:
    print(s)
    if s >= 10000:
        break

8200
10450

This loop stops as soon as it reaches the first day with at least ten thousand steps.

Comprehensions are excellent for building collections, but they are not a replacement for every kind of loop.

You started with a regular for loop that added eight percent tax to a list of prices, and learned how to write the same operation as a list comprehension: an expression, followed by a for clause, inside square brackets.

• An if condition after the for clause filters values, so the result can be shorter than the input.
• A conditional expression with if and else at the front produces a value for every item.
• Curly brackets build a set comprehension, which removes duplicate values.
• A dictionary comprehension provides a key and a value, separated by a colon.
• When a loop's job is to perform an action, or to stop early with break, a regular for loop is the better choice.

Your challenge

Go back to the restaurant menu, with Pasta, Salad and Soup:

menu = {item: price for item, price in zip(items, prices)}
menu

{'Pasta': 12.5, 'Salad': 8.25, 'Soup': 6.0}

Create a new dictionary using a dictionary comprehension. This time there are two requirements:

1. Only include items that cost more than seven dollars.
2. Add eight percent tax to each included item's price.

Think about how to combine a dictionary comprehension with a filtering condition. And remember, the original menu should stay unchanged. Try solving it yourself before looking back at the examples.

The end of the module

This brings you to the end of the module, and it covered quite a lot. You started with lists, then covered tuples, sets and dictionaries, and finally learned how comprehensions let you build all of these with compact expressions.

In the next module, you will learn how to write your own functions.