Often in your programming career, you will inadvertently write flawed code that introduces some fault into your codebase. These faults are called bugs and the activity of fixing bugs is called “debugging.” Of course, as developers, we try to write correct code every time, but writing bugs is simply a fact of life for programmers. The ability to diagnose and fix bugs is the mark of an experienced and talented developer and is foundational for advanced troubleshooting. In this article, we’ll go over some techniques for effective debugging and introduce you to PDB, the Python debugger. At the end of this article, you’ll be ready to not only troubleshoot and fix your own code, but to diagnose and debug existing code as well.
Fundamentals of Debugging
As a developer, you will spend a lot of time debugging. In fact, several sources estimate that as little as 35% or as much as 75% of a developer’s time is spent debugging code that’s already been written rather than writing new code (here’s a pretty good source that puts that number between 35% and 50%). Since so much of our time as developers will be spent debugging, we should put effort into building this skill. As with all skills, you will get better at debugging the more you do it. There are, however, some fundamental techniques to troubleshooting and debugging software that you should know and build upon. This section will introduce those fundamentals, starting with confirming the bug.
Confirm the Bug
An important part of debugging code is understanding the broader context and knowing what the code is intended to do. This is important because your first step in fixing a bug is confirming that it truly is a bug. For example, consider a bug report that says, “the area_of_cube function isn’t returning the proper values unless you pass it 6.” In order to start debugging, we first decide to try the function out. We load up a console and try a few values:
>>> from formulas import area_of_cube
>>> area_of_cube(4)
96
>>> area_of_cube(5)
150
>>> area_of_cube(6)
216It looks right, so we go and double check the formula for finding the surface area of a cube and find that it is the length of an edge squared and then multiplied by six. That means that the numbers we got back from our console test are correct. However, we remember that the bug report says the function only works when you pass it six. Now we wonder if the user reporting the bug is confusing area with volume, since the formula for the volume of a cube is the edge length raised to the third power (or cubed). This means that a cube with edges of length 6 will have both a surface area and volume of 216, and we think the user reporting this bug meant to use the volume function and erroneously reported the area function as having a bug.
At this point, we have to track down the user reporting the bug and see if they really meant to use the area function when they were reporting the bug or if they meant to use the volume function. This is what confirming the bug is about, making sure a reported bug truly is a bug. This is an important step in debugging because it could potentially save you from fixing code that isn’t actually broken. This example may seem silly, but you will be surprised how often reported bugs in the software are actually due to misuse.
Reproduce the Bug
Once you’ve confirmed a bug report is genuine, the next step in fixing a bug is reproducing it. Many times, bug reports will say something isn’t working but when you go to check it out yourself, it seems like it’s working fine. In order to properly diagnose the problem, you have to first see the problem happening. As an example, imagine you get a bug report from a coworker that says something like “The greet_user function isn’t working. The function doesn’t print anything to the console when called.” You start your console to test it out and see no problem:
>>> from greeters import greet_user
>>> greet_user('Erik')
Good evening, ErikIn the above console session, you imported the reportedly buggy greet_user method and called it by passing Erik as an input parameter. The function then printed “Good evening, Erik” as expected. You haven’t yet reproduced the bug, so now you have to go look at the code and see if you can figure out what’s going on. You open up the greeters.py file and look for the greet_user method and see this:
import datetime
def greet_user(name):
now = datetime.datetime.now()
hour = now.hour
if hour < 12:
print(f'Good morning, {name}')
elif hour > 12 and hour < 18:
print(f'Good afternoon, {name}')
elif hour > 18:
print(f'Good evening, {name}')As you look over this code, you see that the method extracts the hour of the day and prints an appropriate greeting: “good morning” if the hour is before 12, “good evening” if it’s after 12 but before 18, and “good evening” if it’s after 18. The bug report said the function isn’t printing anything, how could that be possible?
Upon further inspection, you notice that the method’s if statement only provides instructions for what to do if the hour variable is 0 to 11, 13 to 17, or 19 to 23. If hour is 12 or 18, the code does nothing! This means there’s 2 hours of the day in which this code will not print anything to the console. The person who reported the bug must have been trying to use it during one of those times. In order to test your theory, you decide to manually set the hour variable to 12 to see if that reproduces the bug:
def greet_user(name):
now = datetime.datetime.now()
# hour = now.hour
hour = 12
if hour < 12:
print(f'Good morning, {name}')
. . .Now, you rerun the same commands from your console session earlier:
>>> from greeters import greet_user
>>> greet_user('Erik')But this time, you don’t see any output. You’ve successfully reproduced the bug and you now know why the bug is happening. In order to fix this bug, we simply have to update the conditionals to use greater-than-or-equal-to comparisons:
import datetime
def greet_user(name):
now = datetime.datetime.now()
# hour = now.hour
hour = 12
if hour < 12:
print(f'Good morning, {name}')
elif hour >= 12 and hour < 18:
print(f'Good afternoon, {name}')
elif hour >= 18:
print(f'Good evening, {name}')Now, with your now = 12 line still active, you can see that the code now works. This is why reproducing the problem is important, some of the trickiest bugs only show up under specific conditions.
Challenge Assumptions
Another technique for debugging especially tricky bugs is challenging your assumptions. Sometimes, a bug makes no sense and no matter how many things we test, we cannot seem to figure out why a bug is happening. When this happens, the next step is to ask ourselves if the things we generally take for granted still apply. This means making sure environment variables are what you think they are, verifying any authentication functions are still working, making sure you’re using versions of software that support what you’re trying to do, and so on.
As an example, imagine you receive a bug report saying there’s something wrong with the length_check method; it’s throwing an error any time someone tries to use it. You find the method in your codebase to see it’s defined as the following:
def length_checker(max, values):
if (count := len(values)) > max:
print(f'Cannot have more than {max} values, you have {count}')This method takes two inputs, max and values, and prints an error if the number of items in the values parameter is above the max. Everything seems to look fine to you, so you start a console session to test the method out:
>>> from checkers import length_checker
>>> length_checker(1, [2,2])
Cannot have more than 1 values, you have 2
>>> length_checker(2, [1,2])
>>>In the above example, you call length_checker and pass it a max value of one and a list with two items. Doing so causes the function to print an error to the console as expected. In the next line, we call length_checker with a max value of two and again pass it a two-item list. This time, nothing is printed to the console, as we expected. This function seems to be working perfectly, why did someone report a bug?
It’s time to challenge our assumptions. The error report came from someone else on a different computer, perhaps there’s some difference between their computer and our own that is causing this function to fail. You decide to start by checking the Python version of your computer and the bug reporter’s. You run python --version in your command terminal and see that you’re currently using Python 3.10. The bug reporter runs the same command and lets you know they’re using Python 3.7.
Could the error be because the bug reporter is on an older version of Python than you? You take a closer look at the function and realize that it uses the walrus operator (the := syntax). You vaguely remember reading about this being added to Python a few years ago, so you look up when it was added and find out that the walrus operator wasn’t part of Python until Python 3.8! No wonder the function isn’t working for the bug reporter, that version of Python doesn’t even know what to do with the := syntax. To fix this bug, you simply ask the reporter to update their Python version to 3.8 or higher, and the problem is solved.
Challenging your assumptions is one of the easiest steps to forget when it comes to advanced troubleshooting and debugging. Whenever you find yourself working on a bug that doesn’t make sense, ask yourself if you’ve tested literally every facet of the program—from the operating system to the software versions, and every line of code up to and including the error. Doing so will often show you that one of your assumptions is wrong and that’s why you’re encountering the bug.
Now that we’ve talked about the fundamentals of debugging, it’s time to talk about a built-in Python tool that makes troubleshooting easier: the debugger.
PDB: The Python Debugger
The Python debugger (called PDB) is a tool built into the Python programming language that allows us to stop a script at run time so we can check the value of variables at that given moment in the script’s execution. In this section we’ll learn how to set breakpoints, check the value of variables, and step into functions and methods. We’ll start with setting breakpoints.
Setting Breakpoints and Checking Values
When using a debugger, a breakpoint refers to a line in the code upon which we want to pause execution of the program. Before we see how to set a breakpoint with PDB, let’s write a script in which to use the debugger:
def add(x, y):
sum = x + y
return sum
a = 1
b = 2
c = add(a, b)
print(f"c is {c}")This code is very simple. First, we define a method called add which takes two parameters and returns their sum. Then, we create two variables, a and b, and a third c which is the return value of add when passed a and b variables. Finally, we print the value of c to the console.
Now, lets set our first breakpoint to see how using the debugger works. Just above the line a = 1, write breakpoint(), and then run the script. You should see the following output in the console:
> path/to/script.py(6)<module>()
-> a = 1
(Pdb)There are three things to note in the output above. First, the line preceded with the > symbol is telling us what file we’re in (your output may vary depending on what you named this file and where you saved it). The next line, -> a = 1, shows us where in the Python script we’ve stopped. Notice that this is the line just below where we set our breakpoint. The line a = 1 hasn’t run yet, we’ve paused the script just before its execution. Finally, you see (Pdb) and a prompt. We can type debugger commands in this prompt.
NOTE: You don’t also write import pdb; pdb.set_trace() instead of breakpoint(), but this is considered the “old way”. In fact, if you’re using Python 3.7 or below (which you shouldn’t because they’re all deprecated), you’ll have to use this. When I first wrote this article, I was using the set_trace way exlusively, so if you see one in the article, it’s because I missed it and you can easily just drop in breakpoint().
The first command we’ll run is next, type next into the console and press enter. You should see the following output:
> path/to/script.py(7)<module>()
-> b = 2
(Pdb)Notice that we’re on the next line of the script. The next command executes the line in which we were paused and then pauses on the next line. So, when we typed next just now, Python ran the line a = 1 and stopped on the line you see in the console now, b = 2.
One of the main benefits of using the debugger is seeing the value of variables at a given time. Since we’re paused just after the a = 1 line, we can type the variable name a into the debugger console and see what the value is:
(Pdb) print(a)
1This is useful for when we want to confirm that a variable holds the value we think it does. If we want, we can also change the value of a here as well, check it out:
(Pdb) a = 5
(Pdb) print(a)
5Manually setting variable values at runtime like this is helpful for experimenting when debugging; sometimes you want to see how different values will cause a script to behave.
NOTE: If you have a variable that happens to share a name with a debugger command, you can prefix the variable name with an exclamation mark (!) to have the word interpreted as the script variable instead of the debugger command.
Now, let’s stop pausing the script and allow it to finish running. We do this with the continue command which will let the program run until it reaches the next breakpoint or until it’s done executing. Run continue and you should see the following output:
(Pdb) continue
c is 7Notice two things here. First, the output says c is 7. This is because when we paused the script, we set the value of a to 5, so when we passed a and b to the add function, we actually passed 5 and 2 instead of 1 and 2 like the script appears to say. Also notice that your console is probably back to your normal command prompt instead of the debugger. This is because the script has finished executing.
Now that we know how to set breakpoints, move around in a script, check the value of a variable at a given time, and change the value of a variable, lets look at stepping into functions.
Stepping Into Functions
When we set a breakpoint in a script, the debugger does not leave the scope of where it’s pausing the script. As an example, move your breakpoint from the previous section to just over the c = add(a, b) line. Now, run the script, you should see the following output:
> path/to/script.py(8)<module>()
-> c = add(a, b)
(Pdb)Notice that the debugger is paused just before the add function is called. Type next and hit enter in the debugger console, you should see the following output:
> path/to/script.py(9)<module>()
-> print(f"c is {c}")
(Pdb)Notice that the debugger paused at the next line after the function call instead of going into the add function itself like we might expect. This is because the default behavior of the debugger is to step over function calls so that you, as the developer, can stay inside of the context in which you’re debugging. If you want to step into a function, you have to use the step command. Run the script again, once you get here:
> path/to/script.py(8)<module>()
-> c = add(a, b)
(Pdb)Type step and then press enter. Now, you’ll be in the add function and you should see the following in your console:
--Call--
> path/to/script.py(1)add()
-> def add(x, y):
(Pdb)Notice a couple of new things about the debugger’s output now. First, at the top of the output is a line that says --Call--, this lets us know that we’ve left the context in which we were originally debugging because a function or method call was made. Also notice that the line containing the file location no longer says <module> but says add(). Again, this lets you know that you are currently debugging a function inside of the script.
Now you know how to use the debugger to move around a script, check and change values of variables, and step into functions. These operations exist in virtually every programming language debugger out there, so the commands you learned in this chapter will likely be the same or similar for any other debugger you might use in the future.
Conclusion
In this article you learned the foundational skills of debugging software. We started by learning about three important steps to debugging: confirming the bug, reproducing it, and challenging your assumptions. Then, we learned about PDB, Python’s built-in debugging tool. We learned how to pause a script at run time so we can check the values of variables, then we learned how to control a script’s execution with the next and continue commands. Finally, we learned how to step into functions while debugging with the step command. All of these skills will serve you well in your future work as a programmer and will go a long way in advanced troubleshooting.
If you liked this article about debugging, you may like my primer on TDD with Python.