Formatting, Linting and Type Checking

In the previous lesson, you wrote tests to check whether your code was working correctly, and you saw how those tests could catch a bug even when the program ran without any errors.

Not every mistake needs you to run the code to find it, though. Sometimes you can spot problems just by looking at how the code is written. Examining code for problems without executing it is called static analysis, and this lesson is about the tools that do it for you.

A file with several problems

Here is a new file in the project, src/review_stats/report.py. Its summary function takes your word counts and turns them into a single line of text. The file also contains the first_over function you wrote in the defining functions lesson:

import os
import json
from review_stats.text_tools import top_words
def summary(counts:dict[str,int],n:int=5)->str:
    best=top_words(counts,n)
    unused = len(counts)
    if counts == None:
        return ""
    return ", ".join(f"{word} {count}" for word,count in best)
def first_over(amounts: list[float], limit: float) -> tuple[int, float]:
    for day, amount in enumerate(amounts, start=1):
        if amount > limit:
            return day, amount
print(summary({"great": 5}, "3"))

Running it

Try running this file:

uv run python -m review_stats.report

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "src/review_stats/report.py", line 14, in <module>
    print(summary({"great": 5}, "3"))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "src/review_stats/report.py", line 5, in summary
    best=top_words(counts,n)
         ^^^^^^^^^^^^^^^^^^^
  File "src/review_stats/text_tools.py", line 25, in top_words
    if n < 0:
       ^^^^^
TypeError: '<' not supported between instances of 'str' and 'int'

Immediately you get a TypeError. Python tells you that the less-than operator is not supported between a string and an integer.

Look at the traceback. The last frame points to the top_words function inside text_tools.py. But the actual mistake is not in text_tools.py. It is in report.py. On the last line, you call summary and pass the string "3" as an argument, while the function definition's type hint says that argument should be an integer, n: int. So you have passed the wrong type, and Python only discovered the problem when it tried to use that value.

The other problems

That is not the only issue in this file:

• Unused imports. The file imports os and json, but uses neither of them.
• An unused variable. It assigns a value to unused, but never uses it anywhere.
• Inconsistent formatting. Some lines have spaces in the right places, while others do not.
• A return type that does not match. first_over has a type hint saying it returns a tuple. But what happens if none of the amounts is greater than the limit? The function reaches the end without returning anything, which means Python returns None:

print(first_over([42.1, 18.75, 96.4, 12.0, 130.25], 200))

None

So there are quite a few problems in this one file. The question is: how many of them could you find without actually running the program?

Ruff and mypy

You will use two tools. The first is Ruff. It helps you format your code and detect certain kinds of mistakes. The second is mypy. It checks whether your code is consistent with its type hints. Both are installed as development dependencies in the project, so you run them with uv run.

First, deal with the formatting. Even before you look at the logic, you can see that the code is not laid out consistently. Some spaces are missing, some lines are formatted differently, and the blank lines between functions are not consistent either.

You could go through the file and fix all of that by hand. But a tool can do it for you. That is what a formatterKey termA tool that rewrites code's layout, without changing what it does: ruff format. is for: it automatically adjusts the layout of your code, things like spaces, line breaks and blank lines. The idea is to make the formatting consistent without changing the program's intended behavior.

ruff format

Run Ruff's formatter on the file:

uv run ruff format src/review_stats/report.py

1 file reformatted

Now open the file again:

import os
import json
from review_stats.text_tools import top_words


def summary(counts: dict[str, int], n: int = 5) -> str:
    best = top_words(counts, n)
    unused = len(counts)
    if counts == None:
        return ""
    return ", ".join(f"{word} {count}" for word, count in best)


def first_over(amounts: list[float], limit: float) -> tuple[int, float]:
    for day, amount in enumerate(amounts, start=1):
        if amount > limit:
            return day, amount


print(summary({"great": 5}, "3"))

Look at the summary function. There is a space after each colon and comma. The arrow in the return type has spaces around it. The default argument, n: int = 5, is formatted consistently. And there are now two blank lines between the top-level functions.

So instead of deciding how to format every line yourself, you can let Ruff apply the same style throughout the project. That makes your code much easier to read and maintain.

Formatting is useful, but it does not tell you whether you have written something unnecessary or potentially incorrect. For that, you need another kind of tool: a linterKey termA tool that reads code for mistakes, without running it: ruff check.. A linter examines your code and reports problems it can detect without executing the program. Each kind of problem it checks for is a lint rule, and Ruff includes a linter as well.

Predict the report

Two different jobs

Notice that the formatter did not report any of these problems. It only adjusted the layout. And the linter is not complaining about the spacing you already fixed. That is because formatting and linting are two different jobs. A formatter makes the code's layout consistent. A linter checks for certain patterns that might indicate mistakes or unnecessary code.

Ruff also tells you that some of these issues can be fixed automatically; those are the ones marked [*]. Run the check again with --fix, which tells Ruff to apply the fixes it considers safe:

uv run ruff check --fix

F841 Local variable `unused` is assigned to but never used
 --> src/review_stats/report.py:6:5
  |
4 | def summary(counts: dict[str, int], n: int = 5) -> str:
5 |     best = top_words(counts, n)
6 |     unused = len(counts)
  |     ^^^^^^
7 |     if counts == None:
8 |         return ""
  |
help: Remove assignment to unused variable `unused`

Found 3 errors (2 fixed, 1 remaining).
No fixes available (1 hidden fix can be enabled with the `--unsafe-fixes` option).

Here is the file now:

from review_stats.text_tools import top_words


def summary(counts: dict[str, int], n: int = 5) -> str:
    best = top_words(counts, n)
    unused = len(counts)
    if counts == None:
        return ""
    return ", ".join(f"{word} {count}" for word, count in best)


def first_over(amounts: list[float], limit: float) -> tuple[int, float]:
    for day, amount in enumerate(amounts, start=1):
        if amount > limit:
            return day, amount


print(summary({"great": 5}, "3"))

Ruff removed the unused imports, and with those imports gone, the import-order issue is resolved as well. So Ruff reports two fixes applied, with one issue remaining: the unused variable.

Why the assignment stays

Why did Ruff not just delete that line too? Because removing an assignment is not always harmless. Imagine the right-hand side of that assignment called a function. That function might do something important, even if you never use the value it returns. So automatically deleting the assignment could change the program's behavior. That is why Ruff leaves this particular issue for you to review.

In this example, you know the variable really is not needed, so you can safely delete the line unused = len(counts) yourself.

Now move on to the type hints. Remember the problem at the start: you passed the string "3" to a function that expected an integer, and Python did not complain until the code actually ran. But you already had a type hint saying what the function expected. So should you not be able to catch that mistake earlier?

That is exactly what mypy is for. Recall from the defining functions lesson that Python lets you add type hints to your functions. You can say an argument should be an integer, or that a function should return a string, a list or a tuple. But Python does not enforce those hints at runtime. They are annotations.

That is where a static type checker comes in. A tool like mypy reads those annotations and checks whether the code is consistent with them, and it does that without running your program.

Running mypy

With the unused line deleted, run mypy on the file:

uv run mypy src/review_stats/report.py

src/review_stats/report.py:11: error: Missing return statement  [return]
src/review_stats/report.py:17: error: Argument 2 to "summary" has incompatible type "str"; expected "int"  [arg-type]
Found 2 errors in 1 file (checked 1 source file)

Mypy reports two errors. As with Ruff, the exact wording can vary slightly between mypy versions.

The first error points to the first_over function and says Missing return statement, with the error code return. Why? Look at the function's return type. It says the function always returns a tuple containing an integer and a float. But as you saw at the start, it can also reach the end and return None. That is a mismatch.

The second error points to the call to summary. Mypy says that argument 2 has an incompatible type: you passed a string, but the function expects an integer. That is exactly the mistake you saw when you ran the file.

But notice the difference. When Python raised the TypeError, the traceback ended inside top_words, and you had to follow the calls to figure out where the wrong value came from. Mypy points directly to the call where you passed the string "3". So it can catch this kind of mistake before the program even runs.

Fixing both errors

Start with first_over. The problem is not necessarily that the function returns None. The problem is that the type hint does not say it can return None. So instead of saying the function always returns a tuple of an integer and a float, say it can return either that tuple or None. In Python, you write that using a vertical bar:

tuple[int, float] | None

Now the annotation accurately describes both possible outcomes. While you are there, make the behavior explicit by adding return None at the end of the function. Python would return None automatically if the function reached the end, but writing it explicitly makes the intention clearer.

The second fix is straightforward: instead of passing the string "3" to summary, pass the integer 3, the type the function expects. Here is the file with both fixes:

from review_stats.text_tools import top_words


def summary(counts: dict[str, int], n: int = 5) -> str:
    best = top_words(counts, n)
    if counts == None:
        return ""
    return ", ".join(f"{word} {count}" for word, count in best)


def first_over(amounts: list[float], limit: float) -> tuple[int, float] | None:
    for day, amount in enumerate(amounts, start=1):
        if amount > limit:
            return day, amount
    return None


print(summary({"great": 5}, 3))

Run mypy again:

uv run mypy src/review_stats/report.py

Success: no issues found in 1 source file

And run Ruff's linter as well:

uv run ruff check

All checks passed!

So now the file is consistently formatted, the reported lint issues are gone, and mypy is satisfied with the type hints.

There is another benefit to fixing first_over. Anyone reading that function's signature can now see that it might return None. That is useful information for whoever calls the function, because they know they may need to handle that case.

You have used Ruff to format your code, Ruff's linter to catch unused imports and variables, and mypy to check your type hints. Does that mean your program is now guaranteed to be correct?

Bringing back an old bug

Test that idea. Remember the bug from the debugging lesson, where count_words skipped the first token? Bring it back by changing the loop in text_tools.py so it starts at index 1 instead of index 0:

def count_words(
    tokens: list[str], stop_words: frozenset[str] = frozenset()
) -> dict[str, int]:
    """Return how many times each token appears, leaving out the stop words."""
    counts: dict[str, int] = {}
    for i in range(1, len(tokens)):
        token = tokens[i]
        if token not in stop_words:
            counts[token] = counts.get(token, 0) + 1
    return counts

The tests catch it

Now run the tests you wrote in the previous lesson:

uv run pytest

collected 6 items

tests/test_text_tools.py ..F...                                          [100%]

=================================== FAILURES ===================================
____________________ test_count_words_counts_the_first_word ____________________

    def test_count_words_counts_the_first_word():
>       assert count_words(["great"]) == {"great": 1}
E       AssertionError: assert {} == {'great': 1}
E
E         Right contains 1 more item:
E         {'great': 1}
E         Use -v to get more diff

tests/test_text_tools.py:15: AssertionError
=========================== short test summary info ============================
FAILED tests/test_text_tools.py::test_count_words_counts_the_first_word - Ass...

Of the six tests, one fails immediately (the F among the dots), and the other five pass. The failing one is the test that checks whether count_words includes the first word. Your tests caught the problem that Ruff and mypy missed.

This is an important distinction:

• Formatters keep your code consistently laid out.
• Linters look for certain kinds of problems in how the code is written.
• Type checkers look for inconsistencies between your type annotations and how values are used.
• Tests check whether the code behaves the way you expect, for the cases you have written.

These tools complement each other. None of them replaces the others.

So fix count_words again by going back to looping directly over the tokens:

def count_words(
    tokens: list[str], stop_words: frozenset[str] = frozenset()
) -> dict[str, int]:
    """Return how many times each token appears, leaving out the stop words."""
    counts: dict[str, int] = {}
    for token in tokens:
        if token not in stop_words:
            counts[token] = counts.get(token, 0) + 1
    return counts

uv run pytest

collected 6 items

tests/test_text_tools.py ......                                          [100%]

Now all six tests pass.

Suppressing a warning with noqa

One more thing about Ruff. Sometimes a linter reports something you have written intentionally, and after reviewing it, you might decide you do not want to change that line. Ruff lets you suppress certain warnings with a comment such as # noqa on that line.

But do not use it just to make a warning disappear, because the warning might be pointing to a real problem. Before ignoring it, make sure you understand what Ruff is reporting and why the code should stay as it is. And whenever possible, suppress only the specific rule you intend to ignore, rather than every warning on the line.

Go back to where you started. The file report.py contained several problems. The formatting was inconsistent. There were unused imports and an unused variable. One function's return type did not match what it could actually return. And another function was being called with a string when it expected an integer.

You could have found some of those problems by running the code. Instead, you used tools that examined the code for you:

• ruff format cleaned up the layout.
• ruff check found the unused imports and the unused variable, and with --fix, Ruff automatically fixed the issues it could safely handle.
• mypy checked your type hints and found both the missing return path and the incorrect argument type.

Then you brought back the old word-counting bug, and neither Ruff nor mypy caught it. Only your automated test found that the first word was being skipped.

So you need all of these checks: formatting, linting, type checking and testing. They look for different kinds of problems, and together they help you catch more mistakes before your code becomes difficult to maintain.

Your challenge

This is the final challenge for this module. Run all four checks on the entire project:

uv run ruff format
uv run ruff check
uv run mypy src
uv run pytest

The first formats the Python files in the project. The second checks for lint issues. The third checks the type annotations in your source code. The last runs all your automated tests.

Go through whatever issues these tools report, fix them, and run the commands again. Keep going until the formatter has no more changes to make, Ruff and mypy report no issues, and all your tests pass.

Looking back at the module

You started this module by taking the functions you had written in notebooks and moving them into a Python module. Then you organized that module into a package. You moved the project onto your own computer: you installed Python, created a virtual environment, and learned how to manage your dependencies. Then you worked with actual files, reading reviews from CSV, processing them with your functions, and saving the results as JSON. You learned how to handle exceptions, so one bad input does not have to stop the entire program. You found a bug that raised no exception at all, and learned to reproduce it with a smaller input, inspect variables and step through the code with a debugger. You wrote automated tests to make sure your functions kept producing the right results. And finally, you used Ruff and mypy to catch problems in your code without having to run it.

So you have gone from writing a few functions in a notebook to working with a proper Python project, complete with packages, dependencies, file handling, debugging and automated checks.

Before moving on, think about how you are representing your data. A review is just a string. Its rating is stored separately. Your word counts are stored in a dictionary. Whenever you write a function that works with this data, you pass those values around as separate arguments. That has worked fine so far. But what if you could keep related data together, and not just the data, but also the functions that operate on it? For example, what if a review could hold its own text, product and rating, and give you a way to work with those values?

That is where classes come in. In the next module, you will learn how to combine data and behavior into objects using Python's object-oriented programming features.