Exceptions

Remember the problem from the end of the previous lesson? You had five reviews, and all of them had valid ratings. Now imagine a sixth review comes in, and instead of giving it a rating of 5, someone has written the word "five".

Here is the ratings loop, run on all six ratings:

ratings = ["5", "4", "5", "4", "2", "five"]
total = 0
for rating in ratings:
    total += int(rating)
print(total / len(ratings))

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[1], line 4
      2 total = 0
      3 for rating in ratings:
----> 4     total += int(rating)
      5 print(total / len(ratings))

ValueError: invalid literal for int() with base 10: 'five'

The first five ratings are fine. Python converts "5", "4", "5", "4" and "2" into integers without any problems. But then it reaches the word "five", and when that is passed to int, Python raises a ValueError. The message says invalid literal for int() with base 10: 'five'.

The entire cell stops right there. You do not even get an average rating, even though five of the six rows were perfectly valid.

The questions for this lesson

So here is the problem. How do you tell Python to skip a row it cannot process, instead of stopping everything? And how do you keep track of how many rows were skipped? That is what this lesson covers.

Before fixing the ratings loop, you need to understand what is actually happening when Python encounters an error.

When Python cannot carry out an operation, it can raise an exceptionKey termAn object Python raises when a line can not be carried out; unhandled, it stops the program.. An exception is an object that describes what went wrong. If your code does not handle that exception, Python stops the current execution and shows you a tracebackKey termThe report Python prints for an unhandled exception: the calls that led to it, and the error itself on the last line..

You have already seen plenty of these. The NameError, TypeError and KeyError examples from the earlier modules were all exceptions. So far, you have mostly looked at the last line of the error message. But when you are working with functions, the full traceback becomes much more useful.

A script that fails

Here is a small script called all_tokens.py, in the review analysis project:

from review_stats.text_tools import tokenise


def all_tokens(reviews):
    tokens = []
    for review in reviews:
        tokens += tokenise(review)
    return tokens


reviews = ["Great kettle!", None, "Too loud."]
print(all_tokens(reviews))

Inside it is a function called all_tokens. It takes a list of reviews, loops through them, and calls the tokenise function on each review. Pretty straightforward. But there is a problem with the input: one of the reviews is None.

Run the script from the terminal:

uv run python all_tokens.py

Traceback (most recent call last):
  File "~/projects/review-stats/all_tokens.py", line 12, in <module>
    print(all_tokens(reviews))
          ^^^^^^^^^^^^^^^^^^^
  File "~/projects/review-stats/all_tokens.py", line 7, in all_tokens
    tokens += tokenise(review)
              ^^^^^^^^^^^^^^^^
  File "~/projects/review-stats/src/review_stats/text_tools.py", line 6, in tokenise
    text = review.strip().lower()
           ^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'strip'

You get a traceback. There are several lines here, showing where the error happened and how Python got there.

Following the frames

The next question is: where did that happen? That is what the rest of the traceback tells you. Go back to the top and follow it down.

1. The first frame points to line 12 of all_tokens.py. That is where all_tokens(reviews) is called inside the print statement.
2. The next frame takes you to line 7, inside the all_tokens function. That is where tokenise(review) is called and the result is added to the tokens list.
3. The final frame is line 6 of text_tools.py, inside the tokenise function. That is the line text = review.strip().lower(), and Python highlights the part that failed with ^ marks: review.strip().

So now you know exactly what happened. The tokenise function expected a string, but one of the values passed into it was None. And since None does not have a strip method, Python raised an AttributeError.

Remember the call stack from the recursion lesson? Each function call creates a frame on that stack. A traceback shows you the chain of calls that led to the exception, so you can follow it from the original call, through the functions, all the way to the line that failed.

The same traceback in a notebook

In a notebook, you see the same kind of information, with arrows pointing to the relevant lines:

def tokenise(review):
    text = review.strip().lower()
    for mark in ".,!?":
        text = text.replace(mark, "")
    return text.split()
def all_tokens(reviews):
    tokens = []
    for review in reviews:
        tokens += tokenise(review)
    return tokens
all_tokens(["Great kettle!", None, "Too loud."])

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[2], line 11
      9         tokens += tokenise(review)
     10     return tokens
---> 11 all_tokens(["Great kettle!", None, "Too loud."])

Cell In[2], line 9, in all_tokens(reviews)
      7 tokens = []
      8 for review in reviews:
----> 9     tokens += tokenise(review)
     10 return tokens

Cell In[2], line 2, in tokenise(review)
      1 def tokenise(review):
----> 2     text = review.strip().lower()
      3     for mark in ".,!?":
      4         text = text.replace(mark, "")

AttributeError: 'NoneType' object has no attribute 'strip'

Here the frames are lines of the cell rather than lines of files, but the chain is the same: the call to all_tokens, then the call to tokenise, then the failing line.

So whenever you encounter a traceback, start by reading the exception type and message at the bottom. Then follow the frames to understand how the program reached that error.

Now get back to the ratings problem. You know that converting the word "five" into an integer raises a ValueError. But instead of letting that exception stop the program, you want to handle it. Python gives you a way to do that with try and except.

How try and except work

You put the code that might fail inside a try block. Immediately after it, you add an except block that names the exception you are expecting. That except block is called a handler.

• If the code inside try runs successfully, Python skips the except block.
• If the named exception is raised, Python stops executing the remaining statements in the try block and jumps to the matching except.
• Once that block finishes, execution continues after the entire try statement.

That is what it means to handleKey termCatch an exception with except, so the program carries on. an exception.

The ratings loop, handled

Apply this to the ratings loop. First, create an empty list called kept, which will hold the ratings you successfully convert. Then create a variable called skipped, starting at zero, which will count the rows you could not use.

Inside the loop, put the conversion in a try block: call int on the rating and append the result to kept. Right after that, add except ValueError:, and inside that block, increase skipped by one. That is it.

kept = []
skipped = 0
for rating in ratings:
    try:
        kept.append(int(rating))
    except ValueError:
        skipped += 1
print(kept, skipped)
print(sum(kept) / len(kept))

[5, 4, 5, 4, 2] 1
4.0

The first five ratings convert successfully, so they are added to kept. When Python reaches the word "five", int raises a ValueError. This time, instead of stopping the cell, Python jumps to the except block. skipped goes up by one, and then the loop moves on.

So when the loop finishes, kept contains 5, 4, 5, 4 and 2, and skipped is 1. The average, calculated using only the ratings you kept, is 4.0.

Both problems are solved. The program can skip a row it cannot process, and it can tell you how many rows were skipped.

The same idea is useful when you are asking users for input.

Remember the bill splitter from the while loops lesson? You asked the user how many people were splitting the bill, and kept prompting until they entered a number between 1 and 20. But that version had one problem. What if someone typed four instead of the number 4? When you tried to convert that text with int, Python raised a ValueError, and the program stopped before it could even check whether the number was in range.

Asking until the input is valid

So improve it. Create a function called ask_people. Inside it, use a while True loop, which keeps running until something explicitly ends it. On each iteration, ask the user to enter the number of people. Then, inside a try block, attempt to convert that input to an integer. If it works, return the number.

Remember, return does not just give back a value. It also ends the function, which means the loop ends too.

If the conversion raises a ValueError, catch it. Instead of crashing, print a message telling the user to enter a whole number, and the loop runs again.

def ask_people():
    while True:
        text = input("People: ")
        try:
            return int(text)
        except ValueError:
            print(f"{text!r} is not a whole number, try again")

Try it. Type four as a word, then 4.5, then 4:

ask_people()

People: four
'four' is not a whole number, try again
People: 4.5
'4.5' is not a whole number, try again
People: 4
4

Python cannot convert four, so you get a message saying it is not a whole number and to try again. 4.5 does not work with int either, because you are passing it a string containing a decimal number, so you get the same message. Finally, 4 converts successfully. The function returns 4, and the loop ends.

So now the program can recover from invalid input instead of stopping.

Naming the exception with as

Sometimes you want to know more about the exception that was raised. For that, you can give the exception a name:

except ValueError as error:

Here, error refers to the exception object that Python raised, so inside the except block you can print it. If the input were 4.5:

try:
    int("4.5")
except ValueError as error:
    print(error)

invalid literal for int() with base 10: '4.5'

That is useful when you want to show an error message, log it, or understand exactly why something failed.

A single try block can have more than one except block. That is useful because different operations can fail for different reasons.

Two handlers

Suppose you have three values: the letter "b", the number 0 and the number 2. For each value, you convert it to an integer and divide 1 by it, so inside the loop you write 1 / int(entry).

There are two possible problems here. First, the conversion might fail if the value is not a valid integer. Second, the division might fail if the value is zero. So you add two exception handlers: one for ValueError, and another for ZeroDivisionError.

for entry in ["b", 0, 2]:
    try:
        print(1 / int(entry))
    except ValueError:
        print("not a number:", entry)
    except ZeroDivisionError:
        print("can not divide by zero")

else

There is more to the try statement than try and except. You can also add an else block, which runs only if the try block finishes without raising an exception.

Why is that useful? Because sometimes you want to perform an operation that should happen only after the risky part succeeds.

Go back to the bill splitter. Suppose the bill is 84 dollars and 60 cents, split between four people. Inside the try block, you convert the number of people. Then, in the else block, you calculate each person's share and print the result:

bill = 84.60
for text in ["4", "four"]:
    try:
        people = int(text)
    except ValueError:
        print("not a whole number:", text)
    else:
        print("each pays", round(bill / people, 2))

each pays 21.15
not a whole number: four

With "4", the conversion succeeds and the else block prints each pays 21.15. With the word "four", the conversion raises a ValueError, so the except block runs instead and the else block is skipped.

You could put the print statement inside the try block. But keeping it in else makes the intention clearer: it says this code should run only after the operation in try succeeds.

finally

There is one more part of the try statement, called finally, and it behaves differently. A finally block runs when Python leaves the try statement, whether an exception happened or not. It even runs when the function is returning a value.

Create a function called read_rating. Inside it, try to convert some text into an integer. If the conversion works, return the integer. If it raises a ValueError, return None. Then add a finally block that prints checked, followed by the original text:

def read_rating(text):
    try:
        return int(text)
    except ValueError:
        return None
    finally:
        print("checked", repr(text))

Call read_rating with "5":

read_rating("5")

checked '5'
5

The conversion succeeds, and the function prepares to return 5. But before it actually returns, the finally block runs. So you see checked '5', and then the function returns 5.

Now try the word "five":

print(read_rating("five"))

checked 'five'
None

This time the conversion raises a ValueError. The except block handles it and prepares to return None. But again, before the function returns, the finally block runs, so you see checked 'five', and the function returns None.

So finally is useful for cleanup, or other actions that need to happen whether an operation succeeded or failed.

Remember the with statement you used for files in the previous lesson? It already handles closing the file for you, even when an exception happens inside the block. So for files, you usually do not need to write your own finally block just to close them.

So far, every exception you have handled was raised by Python itself. But you can also raise exceptions yourself, and sometimes that is exactly what you should do. Raising an exception is a way for your own function to say: this input is not valid, so I cannot produce a meaningful result.

A result that looks reasonable

Take an example from the review analysis project. Remember the top_words function? It sorts the words by their counts and returns the top n results, using slicing to do that: something like pairs[:n].

raise

So fix it. At the beginning of top_words, check whether n is less than zero. If it is, raiseKey termCreate an exception and stop the function with it: raise ValueError(...). a ValueError with a message that explains the problem. In code, that is the raise statement: raise ValueError(...), with the message inside the parentheses.

def top_words(counts, n=5):
    if n < 0:
        raise ValueError(f"n must be zero or more, got {n}")
    return sorted(counts.items(), key=lambda pair: pair[1], reverse=True)[:n]

top_words(counts, -1)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[14], line 1
----> 1 top_words(counts, -1)

Cell In[13], line 3, in top_words(counts, n)
      1 def top_words(counts, n=5):
      2     if n < 0:
----> 3         raise ValueError(f"n must be zero or more, got {n}")
      4     return sorted(counts.items(), key=lambda pair: pair[1], reverse=True)[:n]

ValueError: n must be zero or more, got -1

Now the function clearly tells you when it is being used incorrectly. The last line gives the exception type and your message, and the frame above it points at the raise line.

Choosing an exception type

That is a good principle to follow when designing functions. If an argument is invalid, do not silently return something misleading. Raise an appropriate exception with a clear message.

Python already gives you several built-in exception types for this. ValueError is useful when a value is of an acceptable type but is not valid for the operation. TypeError is useful when the argument is of the wrong type. So choose an exception type that accurately describes the problem.

There is one more mistake worth seeing, and it is surprisingly easy to make when you are first learning exception handling.

You might be tempted to wrap a large section of code inside one try block, and then catch every exception with except Exception: followed by pass. The pass statement does nothing. So this tells Python to ignore the exception and continue after the try statement. That might sound convenient, but it can hide serious problems.

Ignoring everything

Suppose you have a list of four values, "1", "2", the letter "x" and "4", and you want to add them together. You put the entire loop inside one try block, and if anything goes wrong, you catch Exception and do nothing:

total = 0
try:
    for text in ["1", "2", "x", "4"]:
        total += int(text)
except Exception:
    pass
total

Catch what you expect

So while exception handling is useful, catching everything and ignoring it is usually a bad idea. A better approach is to catch the specific exceptions you expect, and keep the try block as small as possible.

For example, if you are converting a rating, you know int might raise a ValueError. So you handle that particular exception around that conversion. But if something completely unexpected happens, you generally want to see the traceback, because it might point to a bug in your code that needs fixing.

You started with six reviews. Five had valid ratings, and one had the word "five" instead of a number. At first, that single bad value stopped the entire program. Now you know how to catch the ValueError, skip that row, keep processing the rest, and report how many rows were skipped.

• An exception is an object that describes what went wrong. If it is not handled, Python stops and shows a traceback.
• In a traceback, the last line tells you what went wrong, and the frames above it show how Python reached the failing line.
• try and except handle the errors you expect. A try statement can have several except blocks for different exception types, and Python uses the first one that matches.
• An else block runs when the try block finishes without an exception.
• A finally block runs when Python leaves the statement, whether or not an exception was raised.
• raise lets your own functions report invalid input instead of returning something misleading.
• Catching every exception and ignoring it hides problems. Sometimes an error message is exactly what you need to find a problem in your code.

Your challenge

Improve the bill splitter from earlier. Keep asking the user for the number of people until they enter a valid whole number between 1 and 20.

• If they type something like four or 4.5, handle the ValueError and ask again.
• If they enter a whole number outside the allowed range, like 0 or 25, raise a ValueError yourself, with a message explaining that the number must be between 1 and 20.
• Handle that exception using the same except block.

Whether the user enters invalid text or a number outside the allowed range, the program should explain the problem and prompt them again.

There is one more kind of problem still to come, and in some ways it is harder to deal with. When Python raises an exception, at least you get a traceback, so you know something went wrong. But what if the program raises no exception at all? It runs from beginning to end, prints a result, and everything looks fine, except the result is wrong. In the next lesson, you will learn how to investigate a bug like that, even when Python gives you no traceback to work with.