Debugging in Python, part 2: Developing a debugging mindset

MP 139: A systematic, inquisitive approach makes debugging simple issues easier, and complex bugs possible.

Note: This post is part of an ongoing series about debugging in Python. The posts in this series will only be available to paid subscribers for the first 6 weeks. After that they will be available to everyone. Thank you to everyone who supports my ongoing work on Mostly Python.

One of the most important things you can do to become effective at debugging is to adopt a consistent, systematic approach to every bug you deal with. This mindset will help you debug simple issues quickly, and make it possible to resolve complex issues as efficiently as you can.

Overview

Before getting into the details, let's take a step back and look at the big picture. Much of debugging boils down to these three things:

• How can I recreate the bug?
• What's the root cause of the bug? What's the best way to address the root issue?
• Once it's fixed, how can I make sure this bug won't appear again?

This leaves out a lot of specifics, especially when it comes to handling complex bugs. But these three bullet points outline a clear and consistent mindset when approaching bugs of any size and significance.

Recreating bugs

In order to work on a bug, you need to be able to trigger that bug repeatedly. In order to know that you've fixed the bug, you have to be able to run the project in the same way that produced the bug as well.

If you're working on a single .py file that can only be run one way, you'll probably see the same bug every time you run the program. But most projects can be run in a variety of ways, with a variety of different inputs, and the user can often request different kinds of output. Unless you've run into a serious bug that affects all use cases, you'll probably have to run the project in a very specific way in order to recreate the bug. That's why it's so important to capture as much information as you can when a bug first appears. Some of the information you capture may end up being irrelevant, but you can't always know that ahead of time.

There are a number of specific questions to consider asking in order to help recreate a bug:

• What environment was the project running on?
  • What operating system was in use?
  • What version of Python was used to run the project?
  • How was the project (and its dependencies) installed? What versions of dependencies were installed?
• What was the context of the program's execution?
  • How was the program executed? Was it run at a command prompt, or through an IDE? Was it accessed through a web interface?
  • What CLI options were passed? What configuration settings were used?
  • What data was the project working against?
  • When was the program run?
• What are the actual error messages and problematic output?
  • What was the expected output when running the program?
  • What was the actual output that was generated?
  • If the program crashed, what was the full traceback?
  • Are log files available from the time the bug appeared?
• Can you write a failing test?

The goal in asking these questions is to be able to reproduce the bug in a consistent, reliable way. If you can do so, you can start to identify the cause, work on a fix, and ensure that it shouldn't come up again. The more specific you can be in asking and answering these questions, the more information you'll have when working on the bug.

The ideal way to recreate a bug is to write a test that should pass if the project is working correctly, but fails because of this specific bug. If you can write this kind of test, then you'll be able to prove that you've fixed the bug when the test passes. More importantly, you'll be able to prevent this bug from ever reappearing. Any time this test fails, you won't make a release until it passes again.

Some of these questions are more important for certain kinds of projects. For example if you're working on a bug in a web app, knowing when the bug appeared can be really helpful. You might find that the bug only appears when the app is under heavy usage. For a project that's run independently on a user's system, this kind of information may be less relevant.

Root causes and fixes

Ideally, we'll find the root cause of every bug that comes our way. Many small bugs are symptoms of an underlying issue. If you look for the root cause, you'll not only fix the immediate issue, but save your future self some work as well. Sometimes you won't have time to address the root issue, and will have to just address the current bug that's surfaced. If that's the case, it's still important to document the root cause so you can address it when you have the chance.

When considering the root cause of a bug, there are a number of helpful questions to think about.

What execution path led to this bug?

Through conditional statements, larger projects don't usually execute all the code in the project every time they run. If you can identify which code was running when the bug appeared, you can narrow down the cause of the issue.

Is this the main bug, or is it a symptom of a deeper issue?

Many times a bug is just a symptom of a larger issue in the codebase. The bug we're working on is often just the first reported user-facing manifestation of a deeper issue. If you're working on a bug with a direct cause that only comes up in very specific situations, the fix may be clear, and shouldn't affect any other use cases. But if it's a symptom of a deeper issue, you'll likely see more bugs until you address the root issue.

What's the best fix?

The "best fix" might seem like the one that addresses the root issue. But fixing root causes can often take much more time and resources than we have available at the moment. Sometimes it's quite appropriate to address the immediate issue, while making a plan to address the underlying issue.

What really matters here is documenting what you've learned from dealing with this bug. If you choose to focus on a quick fix, try to do so in a way that will support fixing the underlying issue later. If your quick fix will need to be undone when you address the root cause, make sure you document that clearly. If you take this approach, you can avoid the issue of your codebase growing in complexity from merging a bunch of one-off fixes that each handle specific circumstances.

Merging fixes

Once you've decided on a way to fix the bug, you'll have to merge it into the main branch of the project. Having a clear and consistent approach to this is helpful as well:

• Does the fix affect any other use cases?
• Does the fix need to be released to everyone now, or can it wait for the next planned release?
• Do we need to communicate anything about the bug?

Hopefully your project has a test suite. If your fix addresses the bug and all tests still pass, then you're probably okay to move forward in merging your fix into the main branch of the project. If you don't have a comprehensive test suite, take a moment to consider whether the proposed fix will impact your project in any negative ways.

Some bugs are clearly going to come up for many users until a fix is pushed. If that's the case, the bug might prompt an immediate new release of the project. But many bugs represent edge cases that are worth addressing, but aren't likely to affect many users right away. These kinds of fixes should be merged to the main branch, but don't need to drive a release sooner than you were already planning.

The simplest way to communicate a bug, and its fix, is to make an entry in the project's changelog. This documents the issue and the fix, in a way that anyone interested can access. There may also be a related issue that documents the process of identifying the bug and the fix. But some bugs should be communicated on a wider basis, so people know to watch out for it, and know to upgrade as soon as the fix has made it into a new release.

Bug-reporting issue templates

If you're a GitHub user, you've probably worked with issues at some point. GitHub allows you to write templates that people can use when filing new issues. Writing an issue template for a project takes very little time, especially if you adapt an existing template.

I'm working on a new project called py-bugger. This project lets you intentionally create specific kinds of bugs in a working project, in order to practice debugging. We'll use py-bugger a little later in this series. As people start to use this project, I'd like to capture the right kind of information any time it fails to introduce the expected kind of bug in a user's project. Here's what you'll see if py-bugger doesn't work as it should, and you decide to report the issue:

Rather than just entering information into a blank issue, users are prompted for exactly the kind of information that should help address bugs in this project. This is a project-specific adaption of what was discussed in the first part of this post.

If you want to see what the rest of the template looks like, go to the main Issues page for py-bugger and click New Issue. Then click Bug Report, and you should see the full blank template. If you're interested in the raw template, you can find it in the .github/ISSUE_TEMPLATE/ directory. The template file is named bug_report.yml.

This kind of template isn't just for external users. If you're working on your own project, you might not need to fill out the full template. But noting the kind of information that's prompted for in a template like this is critical in addressing all but the simplest bugs. If you struggle with debugging because you don't yet have a disciplined approach to managing the kind of information that supports efficient debugging, pushing yourself to use a template like this can go a long way toward developing a more consistent and effective process.

One more thing: while I highly recommend starting with an existing template, I also encourage people to adapt these templates to their own projects and workflows. If you have an important project that only you use, maybe you can leave out the environment section. If you're working on a data analysis project and the target dataset is always changing, maybe you need a widget for uploading the data that was used when the bug was encountered. Consider what's most useful when approaching bugs in the projects you work on, and develop templates that consistently capture exactly that information.

Conclusion

When people think about debugging, they tend to think of diving into a codebase to figure out what isn't working. But if you want to work efficiently and effectively, there has to be an information-gathering phase before you dive into the code. If you've got a fair bit of experience, the information-gathering doesn't always need to be formal. But if you're new, or if you're facing a complex bug, or if you're solving someone else's bug, a clear and consistent approach to gathering the right kind of information when it's available is critically important. Considering the questions discussed in this post should set you up well for tackling the next phase, focusing on the code that needs to be updated.

In the next post, we'll tackle a specific bug. We'll start simple, by debugging a single-file project. But even so, we'll run through the steps outlined above, in order to establish a consistent approach to handling any new bug that shows up.