Python 3.10 Introduces better error messaging

Herbert van Leeuwen/
09 September, 2021

A few weeks ago, we took a look at pattern matching in Python 3.10. In this blog, we dive into the improved error messages.

Python 3.10 comes with some significant quality of life improvements that will make debugging and documenting your code more manageable. When I heard about these improvements, I began by reading prerelease notes for Python 3.10. There I was pleasantly surprised by changes that have been made to an old friend, the SyntaxError. As a beginner programmer, I was often greeted by this error message. If Google is to be believed, there are still a lot of programmers googling this same message.


Google trends data showing that "Python Syntax error" remains a relevant search term

In previous versions of Python, most syntax errors came with ambiguous messages like 'SyntaxError: invalid syntax' and 'SyntaxError: unexpected EOF while parsing' pointing to an incorrect location. With Python 3.10, many new specialized error messages are introduced to provide better feedback. Let's take a look at these improved error messages.

The code below is used to select cities with a high population. Unfortunately, the code has multiple syntax errors.

def select_big_cities(city_population: Dict):
    """Select all cities with more than a million inhabitants and returns a list of tuples"""
    return [city, pop for city, pop in city_population.items() if pop > 1000]

city_population = {'Amsterdam': 821, 'Brussels': 174, 'Stockholm': 975,
                   'London': 8982, 'New York': 8419, 'Paris': 2161

big_city_population = select_big_cities(city_population)

In Python 3.9, this code fails with:

File "blog_39/blog.py", line 8        
    return [city, pop for city, pop in city_population.items() if pop > 1000]
                      ^
SyntaxError: invalid syntax

We get the familiar "SyntaxError: invalid syntax" message and are pointed to the keyword "for" in the error message. This does not really help us to understand the error in our code. Did I misspell one of the keywords? or should my conditional be earlier in this statement?
In Python 3.10, this code results in:

File "blog_310/blog.py", line 8
    return [city, pop for city, pop in city_population.items() if pop > 1000]
            ^^^^^^^^^
SyntaxError: did you forget parentheses around the comprehension target?

In this case, we get an unambiguous error message about adding parentheses around the comprehension target. The error message also points us to the location in the list comprehension where the error occurred. We can fix the code more easily.
Now with that error fixed, Python 3.9, complains about:

File "blog_39/blog.py", line 14
    big_city_population = select_big_cities(city_population)
    ^
SyntaxError: invalid syntax

While Python 3.10, suggest:

File "blog_310/blog.py", line 14
    city_population = {'Amsterdam': 821, 'Brussels': 174, 'Stockholm': 975,
                      ^
SyntaxError: '{' was never closed

Now that is what I call a time saver! We forgot to close the curly brackets when defining the dictionary and the stack trace even points to the bracket that has not been closed.

Another example would be if we were to remove one of the commas in the dictionary definition. Like so:

city_population = {'Amsterdam': 821, 'Brussels': 174, 'Stockholm': 975,
                   'London': 8982, 'New York': 8419 'Paris': 2161}

In Python 3.9 this results in:

File "blog_39/blog.py", line 12
    'London': 8982, 'New York': 8419 'Paris': 2161}
                                     ^
SyntaxError: invalid syntax

In Python 3.10 this results in:

File "blog_310/blog.py", line 12
    'London': 8982, 'New York': 8419 'Paris': 2161}
                                ^^^^^^^^^^^^
SyntaxError: invalid syntax. Perhaps you forgot a comma?

Python 3.10 noted - in exquisite British fashion - That PERHAPS we forgot a comma. Which is exactly what we did. Python even highlights the two items in between which the comma should be placed.

Some other notable specialized Syntax error messages are related to:

The new specialized syntax error messages can clearly help us quickly find and fix our syntax errors, but these are not the only improved error messages in the Python 3.10 release.

IndentationErrors are improved with more context including the location where the indentation is wrong.

AttributeErrors offer suggestions if you misspell an attribute. It does this by looking at the similar attributes in the object the AttributeError was raised from, for example:

class Math:
    sqrt = 0
    squirrel = "squirrel"

m = Math()
m.sqrr

File "<stdin>", line 1, in <module>
    m.sqrr
AttributeError: module 'Math' has no attribute 'sqrr'. Did you mean: 'sqrt'?

Actually, we meant to use the squirrel attribute. But this does show us how these new helpful messages provide us with the most similarly named attribute.

Finally NameErrors also offer suggestions of similar variable names. For example:

List_of_numbers = [1, 2, 3, 4, 5]
List_of_numburs.remove(5)

File "<stdin>", line 1, in <module>
    List_of_numburs.remove(5)
NameError: name 'List_of_numburs' is not defined. Did you mean: 'List_of_numbers'?

Note: The improvements for Attribute and NameErrors don't work in certain REPLs (e.g., IPython) because they don't call PyErr_Display() to display the error.

Conclusion

Python 3.10 greatly improves the ergonomics when using the language and we cannot wait for it to be actually released. If you want to learn more about these and other changes to Python in version 3.10, I highly recommend you read the prerelease notes and the accompanying PEP's.

Subscribe to our newsletter

Stay up to date on the latest insights and best-practices by registering for the GoDataDriven newsletter.