Don't stringify Python exceptions with `str(e)`

Sometimes I see Python code that tries to deal with exceptions like this:

raw_input = ...

try:
    output = get_output(raw_input)
    print(output)
except Exception as e:
    print(
        "Something went wrong: " + str(e),
        file=sys.stderr
    )

Upon encountering an exception, it prints it in an error message. In principle, that’s fine. The problem is that it does it with str(e). The output of str(e) (or, equivalently, the f-string f"{e}") can be surprisingly useless.

To contrive an example, suppose this is the part of the code that might raise the exception. It parses some values out of a JSON object.

# Example input:
#   '{"first_name": "Jane", "last_name": "Doe", "age": 64}',
# Example output:
#   "Jane Doe is 64 year(s) old."
def get_output(raw_json_string: str) -> str:
    parsed_json = json.loads(raw_json_string)
    first = parsed_json["first_name"]
    last = parsed_json["last_name"]
    age = parsed_json["age"]
    return f"{first} {last} is {age} year(s) old."

Then suppose we feed it a JSON object that’s missing one of the expected keys. For example, suppose we supply {"first_name": "Jane", "last_name": "Doe"}, which is missing the age key. That’ll cause this code to raise KeyError("age").

The calling code will catch that KeyError, pass it through str(e), and finally print…

Something went wrong: age

Which is…very confusing! That’s literally it, too. str(e) for a KeyError will literally only return the key that was missing. It doesn’t even tell you it was a KeyError.

I’m sure that with a healthy amount of pessimism, you can see how this would be a big WTF in the real world. Can you imagine getting a bug report like this? “The ‘age’ of what? From where? What was wrong with it?” Who knows. I have had to debug real occurrences of this. It’s not fun.

What to do instead

Use the helpers from traceback

Ideally, we want full exception information available to us, like what Python outputs by default when an exception goes unhandled:

Traceback (most recent call last):
  File "test.py", line 123, in <module>
    output = get_output(raw_input)
  File "test.py", line 456, in get_output
    age = parsed_json["age"]
          ~~~~~~~~~~~^^^^^^^
KeyError: 'age'

str(e) can’t do that, but the helpers from the standard traceback module can.

The closest thing to a direct replacement for str(e), taking an Exception argument and returning a str, is to use traceback.format_exception() like this:

except Exception as e:
    message = (
        "Something went wrong:\n\n"
        + "".join(traceback.format_exception(e))
    )
    print(message, end="", file=sys.stderr)

The API is a little weird. It returns “a list of strings, each ending in a newline and some containing internal newlines.” So, we need to concatenate them with "".join(...); and we need to pass end="" to print(), to stop it from adding a redundant newline at the end. But this otherwise does what we want!

Something went wrong:

Traceback (most recent call last):
  File "test.py", line 123, in <module>
    output = get_output(raw_input)
  File "test.py", line 456, in get_output
    age = parsed_json["age"]
          ~~~~~~~~~~~^^^^^^^
KeyError: 'age'

Other functions in traceback give structured access to the different parts of this output, so you can pare it down or customize it, if you really want. See the traceback docs for details.

Use the helpers from logging

If you happen to be sending this output to the standard logging module, it can do the job for you.

Pass an exc_info argument to methods like log.error() and log.warning(), and they’ll automatically include exception information along with your message. Like this:

import logging

_log = logging.getLogger(__name__)

try:
    ...
except Exception as e:
    _log.warning("Something went wrong.", exc_info=e)

Which will print something like this (depending on your logging configuration):

WARNING:test.py:Something went wrong.
Traceback (most recent call last):
  File "test.py", line 123, in <module>
    output = get_output(raw_input)
  File "test.py", line 456, in get_output
    age = parsed_json["age"]
          ~~~~~~~~~~~^^^^^^^
KeyError: 'age'