Testing a book's code, part 2: Basic scripts
MP #77: Using parametrization to efficiently write a large batch of tests.
Note: This is the second post in a series about testing. This post, and the ones that follow will only be available to paid subscribers for the first 6 weeks. After that they will be available to everyone. Thank you to everyone who supports my ongoing work on Mostly Python.
In the previous post, I laid out an overall plan for testing all the code in the book. In this post, we’ll put that plan into action. We’ll make a fork of the book’s repository, and build a test suite in the new repository. We’ll do this in a way that allows us to pull in changes from the original repository, without any conflicts with the testing code.
We’ll also use parametrization to write a set of tests that cover all the basic scripts in the first half of the book. These tests will run each script that should be tested, and make sure the output is what we expect it to be.
You can follow along and run the code yourself, if you’re interested in doing so.
Fork the original repository
Making a private fork of a public repository is not as straightforward as it sounds. GitHub doesn’t automate the process. I believe that’s partially for technical reasons, and partially to encourage public forks. The best resource I found was this gist, and this official post about forking.
I’m going to set up a public fork for this post, called mp_testing_pcc_3e. You can’t make a direct fork of your own repository; I ended up cloning the original pcc_3e repo, and pushing it to a new repo. If you want to follow along, you can simply make a fork of the original pcc_3e repository.
Testing basic programs
We’ll start by testing the basic programs in the book. These are programs that don’t use any third party libraries. They don’t write or modify any files, and they don’t accept user input. All we need to do is run them, and validate the output.
We’ll start by testing the first program in the book, hello_world.py. Just as a typical Hello World program demonstrates that your environment is set up correctly, this first test will show that our testing infrastructure is set up correctly and that a simple test can pass.1