Making sure the software we build works the way we (and our customers) want it to work is called, unsurprisingly, software testing. Software testing is an enormous topic; indeed, there are entire books, courses, conferences, academic journals, and more about the topic. One can even make a career out of testing software. We couldn’t possibly scratch the surface of the complexity involved in software testing in this article, so we’ll only focus on a topic most relevant to us as programmers: test-driven development.
What is Test-Driven Development?
To put it simply, test-driven development, or TDD, is a programming style in which a component’s tests are the primary mechanism guiding its development–this is often accomplished by writing tests before writing feature code. This is in contrast to the way software is traditionally built, where a feature is built end-to-end and then testing is added later as an afterthought.
There are several reasons for taking a test-first approach to building software. Most importantly, it forces you to think before you start writing code. Pondering how you might test something you’ve not written yet will necessarily make you imagine how you intend to build it, what you want it to do, and how other parts of the code might interact with it. Writing tests first also helps flush out unanswered questions you hadn’t yet thought about (for example, should a math function gracefully handle bad datatype inputs or fail loudly? What kind of data should a function expect as input? And so on).
One practical reason for writing tests before application code is that after a while, you have a fairly comprehensive test suite that reports on the functionality of most—if not all—of your system. This is a lifesaver as your projects get more complex and the possibility of a minor change in one part of the code breaking a feature in another area increases.
Now you may be wondering what it means to “write” a test. In the context of TDD, the word test is almost always shorthand for unit test—a program that checks the functionality of one small piece of functionality (a unit). If that sounds like it means we write code to test code, it’s because we do! In Python, unit tests are classes and methods that we build in order to verify the performance of other classes and methods that we build. Let’s first learn how to write unit tests before diving in to the TDD workflow.
Getting Started with Unit Testing in Python
As their name suggests, unit tests test one unit of code—that could be a function’s return value, the attributes of a class, or even a single unit of functionality like the ability to login to a website. We don’t have to get hung up on the semantics right now, let’s just start writing. Create a file called add_fucntion.py and write this function in it:
def add(x, y):
return x + yThe above function simply takes two arguments and returns their sum (adds them together). Now, suppose we wanted to test this function to make sure it works. We might open an interactive Python shell and make sure it returns the values we expect based on the values we pass to it:
>>> from add_function import add
>>> add(5, 5)
10
>>> add(4, -2)
2
>>> add(True, "hello")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "path/to/add_function.py", line 2, in add
return x + yIn the above session, we imported the add function from add_function. We passed 5 and 5 to the function to see if we got 10 as expected; we pass a 4 and a -2 to make sure the function handles signed and unsigned integers appropriately; and finally, we pass the Boolean True and string hello to the function to make sure it throws an error. All inputs behave as expected, so we can be pretty sure the add function behaves as expected.
This is fine for this function since it’s small and simple, but what if we have much larger functions or functions that change a lot? Are we going to manually test every function in a Python shell like this? As you might expect, the answer is no!
This is where unit testing comes in, we’ll write a program that will run all these different lines and more. Python has two popular testing frameworks: pytest and unittest. The unittest module is part of the standard library, meaning we don’t have to do anything special to install it, so we’ll use it for the examples in this chapter.
To get started with unittest, make a file in the same directory as add_function.py and call it test_add_function.py. Inside this new file we first need to import the unittest module and the file with the add function in it. To do that, make the top of the test_add_function.py file look like this:
import unittest
from add_function import addNow, with the add function and unittest module imported, it’s time to write our first test. With unittest, we start by first defining a test class. Test classes are usually a collection of tests for one conceptual level of functionality; in this case, the add function. Let’s call ours AddTest:
class AddTest(unittest.TestCase):Notice that the test class inherits the TestCase class from unittest. This gives us access to the assert methods we’ll use in a few minutes. Next, we’ll define a test method. Test methods are what make up test classes and we usually write one for each of the different ways our code might be used. For example, the most likely use of our add function is probably to add positive numbers, so let’s write a test method for that:
class AddTest(unittest.TestCase):
def test_positive_addition(self):In the above code, all we did was add the test method test_positive_addition to the AddTest test class. Now we’re ready to write the actual assertions. In this case, we’ll pass the add function two numbers, save its return value in a variable called actual, and compare that with a variable called expected (the number that should be returned). Then, we’ll use the assertEqual method from unittest to test that the two values are the same. Here’s how your entire test_add_funciton.py file should like by now:
import unittest
from add_function import add
class AddTest(unittest.TestCase):
def test_positive_addition(self):
expected = 30
actual = add(10, 20)
self.assertEqual(actual, expected)Notice inside of the test method test_positive_addition we create two variables, expected with the value of 30, and actual with the return value of add when passed 10 and 20. Finally, we call the assertEqual class method and pass it actual and expected. As you might guess from its name, this method checks if the two values passed to it are equal.
You’ve now written your first unit test! It’s time to actually run the test. Open a terminal and, from whatever folder your add_function.py and test_add_function.py files are in, run the following command:
python -m unittestThis runs the unittest program which will find any file that starts with the word test and runs the assertions inside. You should see output similar to the following:
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OKYou might not see it, but the single dot at the top of the terminal output represents a test method that passed. Had the test failed, you would have seen an F. This segues nicely into an important point: you should always make sure your tests can fail. We actually never saw our test fail so how can we be 100% certain that our unit test is working?
To make sure our unit test is working the way we expect, let’s change the expected variable to 40 and see what happens. Make this change in your code:
expected = 40And run python -m unittest in your console once again. You should see something like the following output:
F
==================================================================
FAIL: test_positive_addition (test_add_function.AddTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/path/to/test_add_function.py", line 9, in test_positive_addition
self.assertEqual(actual, expected)
AssertionError: 30 != 40
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (failures=1)This is very interesting output, not only do we see that our test is testing the right thing (which we can see by the FAILED message when we tried to see if 20 + 10 could equal 40), but we also see what it looks like when a test fails. Notice that the output includes the file name and test method that failed as well as the line number of the assertion that caused the failure. This is super helpful when running big test suites (collections of automated tests like this one) with hundreds or thousands of tests.
Notice too that the failure message also tells us what failed. Specifically, it tells us that 30 != 40, which is exactly what we put into the test to force a failure. Go ahead and change the expected variable back to 30.
Now that we know how to write unit tests, let’s write a couple more. First, let’s test the method’s ability to handle negative numbers. We’ll also write this test specifically to fail. Check it out:
def test_negative_addition(self):
expected = 0
actual = add(10, -20)
self.assertEqual(actual, expected)Here we’ve defined another test method under the AddTest test class. This time, we are using negative numbers to see if the add function handles numbers lower than zero. We’ll test this by passing 10 and -20 to the add function, but instead of setting the expected variable to -10, we set it to 0. We do this because we want to make sure the unit test we just wrote is testing the right thing, and the best way to do that is by forcing an assertion failure. Once again, go ahead and run python -m unittest in your terminal, you should see something like the following:
F.
==================================================================
FAIL: test_negative_addition (test_add_function.AddTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "path/to/test_add_function.py", line 14, in test_negative_addition
self.assertEqual(actual, expected)
AssertionError: -10 != 0
----------------------------------------------------------------------
Ran 2 tests in 0.001s
FAILED (failures=1) Notice this time that the first line of output has a . followed by an F. This means that one of the test methods failed while another passed. As we hoped, the rest of the output tells us that our test_negative_addition method failed because -10 does not equal 0. Go ahead and change the expected variable to -10 so that the tests will pass.
Let’s write a slightly different test now. In the previous examples, we asserted two values were equal. This is of course an important kind of test to run but that’s not the only thing functions ever do. Remember earlier we tested what would happen by calling the add function by passing it True and the string hello and we saw that it raised a TypeError? We can actually test for that behavior too, check it out:
def test_bad_datatypes(self):
with self.assertRaises(TypeError):
add(True, "hello")In the above test method, we start by using the with keyword in combination with the unittest assert method assertRaises and pass it TypeError. Finally, inside the with block, we call the add function and pass True and the string hello. We have to use the with block because otherwise the method will error (even though it’s the error we’re testing for).
If you run this, it should pass. For the sake of time, I did not show you how to make this test intentionally fail. I leave that as an exercise to the reader.
Aside from assertTrue and assertRaises, unittest has many other assertion methods such as assertAlmostEqual for approximating, assertIn for checking the presence of a single element in a list, assertFalse for checking things you expect to be untrue, and many more.
Now that we’ve learned the basics of writing unit tests in Python with unittest, let’s talk about the flow of TDD.
The TDD Flow
As I mentioned at the beginning of this article, test-driven development often consists of writing tests before writing actual code, but that’s just the beginning. The TDD workflow follows a three-phase cycle: write a failing test, write just enough code to make that test pass, refactor. You repeat this cycle until the code is doing what it’s meant to do, is thoroughly tested, and is as well designed as it can be. To understand the TDD workflow, we’ll start with analyzing the requirements and then repeating the cycle a couple of times.
The Requirements
Imagine we are building a cash register application for our favorite coffee shop. This application currently consists of a Drink class, representing the many different drinks the coffee shop serves, an Order class that records a customer’s orders, and an OrderItem class representing an individual item in an order. These three simple classes are in a file called coffee_cash_register.py and look like this:
class Drink:
def __init__(self, name, price):
self.name = name
self.price = price
class OrderItem:
def __init__(self, drink):
self.drink = drink
class Order:
def __init__(self, items):
self.items = items
def add_item(self, item):
self.items.append(item)The above class hierarchy consists of a Drink class which contains just a constructor and name and price attributes, an OrderItem class which simply contains a drink attribute, and an Order class that contains a list of OrderItem instances and a method for adding more OrderItem objects to that list.
We now want to build a class called CashRegister that serves as an way to access the Order class and allows an employee to create orders and build receipts. Since we are doing this the TDD way, our first step is to write a failing test for the CashRegister class.
Write a Failing Test
To write our first test for CashRegister, we first have to think about what kind of attributes and features the CashRegister class should have. Let’s start by thinking about what a real-world cash register does. It allows an employee to build an order by adding items a customer wants to purchase or removing items if one was accidentally added to the order. It should also be able to calculate the total price for the entire order, so the employee knows how much to charge the customer.
This thought process has revealed three things the CashRegister class must be able to do, and thus three test methods. Create a file called test_cash_register.py and write the following code:
import unittest
from coffee_cash_register import *
class CashRegisterTest(unittest.TestCase):
def test_add_item_to_order(self):
cr = CashRegister()
def test_remove_item_from_order(self):
cr = CashRegister()
def test_calculate_total(self):
cr = CashRegister()Now remember, you want to write just enough of the test that the test should fail. We currently have three test methods, each creating a new instance of the CashRegister class—a class we haven’t even created yet. Run python -m unittest in the directory you’ve created these files and see that you have three failures with messages like name CashRegister is not defined. We’ve successfully completed the first part of our TDD cycle, writing a failing test.
Get the Test to Pass
Now that we have our test class and three test methods created and failing, it’s time to get them to pass. Since our tests failed because CashRegister wasn’t defined, let’s go ahead and create that class. In coffee_cash_register.py, add the following class:
class CashRegister:
def __init__(self):
passHere, we’ve created the CashRegister class with a very basic constructor. If you run the tests again, you’ll see that they pass. Believe it or not, that means we’re actually done with the second part of the TDD cycle: getting the test to pass. Let’s move on to the next step.
Refactor
Before we return to the start of the TDD cycle, we need to refactor the code we’ve written. This includes both the test code and the application code; but right at this moment in our particular example, the only thing that needs refactoring is our test code. Notice how we instantiated a CashRegister object at the beginning of each test method. In order to save us from retyping the same line for every test method, let’s just make cr a class attribute:
class CashRegisterTest(unittest.TestCase):
cr = CashRegister()
def test_add_item_to_order(self):
pass
def test_remove_item_from_order(self):
pass
def test_calculate_total(self):
passNotice that we defined an attribute for the test class called cr and instantiated it with CashRegister(). Now we don’t have to write this line at the beginning of every test method. Notice also that we replaced the line inside each test method with pass, this is simply because since we no longer need to instantiate a CashRegister object, there is nothing to write in the test methods. If we leave them empty, our subsequent test runs will fail.
Now, let’s repeat the TDD cycle a couple more times to really get a feel for how this works.
Repeat
We’ve now completed one iteration of the TDD cycle. The next step is to write another failing test. Actually, we’ll just add to one of our existing tests and get it to fail. Let’s write the test code for the test method test_add_item_to_order. Again, we have to start by thinking about how this might work. We decide that adding an item to an order will most likely be done by creating a method in CashRegister that takes an instance of Drink, turns it into an OrderItem, and adds it to an Order object’s items attribute. We write the test method as such:
def test_add_item_to_order(self):
drink = Drink("Latte", 3.49)
self.cr.add_item_to_order(drink) Now, if we run the tests, we should see a failure from this test method because the CashRegister object has no attribute add_item_to_order. The next step in the cycle is to get the test to pass, which right now means building that method for the CashRegister class. We might start by writing a method in CashRegister that would look something like:
def add_item_to_order(self, drink):
item = OrderItem(drink)But now that we’ve written this line and are wondering how to add it to an order, we realize that neither the method nor the class actually has an Order object to add the OrderItem to. Now we have to think about how we plan to instantiate the idea of an Order into our CashRegister app. Do we make it a class attribute of the CashRegister class? Do we make CashRegister inherit Order? Neither of these approaches sound quite right since a cash register in real life is mostly independent from the orders it builds. We decide instead to pass an instance of Order to the add_item_to_order method. Check it out:
def add_item_to_order(self, drink, order):
item = OrderItem(drink)In the method above, we’ve updated the argument list to take in an order parameter—an instance of the Order class. This will allow us to add the OrderItem instance to an actual Order instance. We also need to update our test code to account for the new method parameter:
def test_add_item_to_order(self):
drink = Drink("Latte", 3.49)
order = Order([])
self.cr.add_item_to_order(drink, order)Now that we’ve instantiated an empty Order object, we can call the add_item_to_order method without breaking anything. This means we’ve completed the “get test to pass” portion of the TDD cycle once again! The next step should be to refactor the code but, in this case, there is not really much to refactor, so we start the cycle over once again.
We’re now starting iteration three of the TDD cycle and once again, our first task is to get the test to fail. The best way to accomplish that at this point is to write our first assertion. Since the add_item_to_order method is supposed to add an Item object to an Order object’s items attribute, and since we are instantiating an Order object with an empty items attribute in our test, we can assert that this method is working as intended by checking that the length of order.items is greater than 0. Check it out:
def test_add_item_to_order(self):
drink = Drink("Latte", 3.49)
order = Order([])
self.cr.add_item_to_order(drink, order)
self.assertTrue(len(order.items) > 0)Notice the line we added, self.assertTrue(len(order.items) > 0). We figure if the add_item_to_order method is working like it should, the order.items list will have one item in it. If we run this test, it should fail with a message like False is not True. This fails because, in our application code, we never add the OrderItem object to the Order object’s items attribute. We’re again done with the first part of the TDD cycle. Let’s complete the second part of the TDD cycle by fixing the add_item_to_order method:
def add_item_to_order(self, drink, order):
item = OrderItem(drink)
order.add_item(item)We make this test pass by adding a call to the Order object’s add_item method which adds the item variable to the order variable’s items attribute. If you run the tests again, you should see that this one now passes. It’s time for the refactor stage.
Now that we think about it, the test we wrote is a little weak. The length of the order variable’s items attribute is only a side effect of what we really wanted to do which was add a LineItem containing the drink variable to the order variable’s items attribute. By only testing the length of items, the add_item_to_order method could add anything at all to the items attribute and our test would say that it’s passing. For the refactor stage, let’s be a bit more explicit about what we’re testing:
def test_add_item_to_order(self):
drink = Drink("Latte", 3.49)
order = Order([])
self.cr.add_item_to_order(drink, order)
item = order.items[0]
self.assertIsInstance(item, OrderItem)
self.assertEqual(item.drink, drink)We’ve rewritten the test method to be more explicit. First, we extract the first object out of the order variable’s items attribute after calling the add_item_to_order method. We expect that the item should be an OrderItem object, so we use the unittest assertIsInstance method to make sure it is. Then, we use the assertEqual method to make sure that the item variable’s drink attribute is the Drink instance we created earlier in the test. This is a much more explicit test of the functionality we want because not only will it tell us if the method incorrectly created the OrderItem, it will let us know if we somehow accidentally appended the wrong kind of object to the Order instance’s items list.
It looks like this part of the class is pretty thoroughly tested now and we’re confident that the CashRegister class’s add_item_to_order method works as expected. You’ve now completed your first TDD workflow! Practice your understanding of this flow by repeating the steps for the other two test methods. I’ll give you a hint, to write a failing test for test_remove_item_from_order, you’ll probably have to call a method you haven’t written yet.
Conclusion
This article gently introduces the world of software testing by exploring the concept of test-driven development (TDD). We started by defining what TDD is and why it’s helpful. We highlighted the fact that TDD not only helps us think about the software we build, but it also has the side effect of leaving us with a collection of tests that we can always check our software against. Then, we took a quick detour to talk about how to use Python’s unittest testing framework so we could learn what it means to “write” tests. We did this so we could practice the TDD workflow with an example. In our example we ran through the TDD cycle three times to fully build out one thorough test method for our CashRegister class.
To learn more about TDD, I suggest you start trying to use the flow in your daily programming work. At first, you will likely be less productive than you are used to, but as you practice more, you may be more productive than you’ve ever been! If you are primarily a Python programmer, I suggest you install pytest and learn how to use that instead of unittest. The pytest package is much more popular in Python projects, we started with unittest so we could skip the tedium of installing a new package.
Comments
One response to “Test-Driven Development with Python: a Primer ”
[…] If you liked this article about debugging, you may like my primer on TDD with Python. […]