The Python programming language not only shines in fields such as machine learning and data analysis, but also has an increasing number of users in web development and software development.
In software development, there is a advocated development paradigm: test driven development. In this development paradigm, writing unit tests is essential. If strict test driven development is not implemented, writing unit test programs is also necessary.
For unit testing, the most basic module is pytest, which will be briefly introduced in this article. In addition, there is a currently popular module called fizz buzz, which will also be recommended to readers in this article.
Why automated testing is necessary
Not everyone understands the necessity of automated testing, and some even consider it a pure burden. They believe that when writing code, they have already discovered bugs in the program and have fixed them in a timely manner.
That's not entirely unreasonable, because during development, we write code and execute programs at the same time. If there are any problems, we will definitely make timely modifications. Especially for programmers with rich development experience, there are very few errors in the code they write.
However, bugs are inevitable. Generally, we use existing frameworks or libraries for development, rather than writing every line of code from scratch. It is also possible to maintain, modify, or upgrade existing features. In these cases, the probability of bugs appearing in the program is even higher.
Therefore, automated testing is indispensable. Developers should consider automated testing as a safety net for their code to prevent bugs from occurring due to the addition of new features.
Another reason for implementing automated testing is that manual testing can sometimes be difficult to complete all functional testing of a program. For example, a program obtains some data from a third-party API, and if manual testing is used, it cannot test the abnormal information obtained by the other service when a problem occurs. However, if automated testing is used, it can be easily implemented.
Unit testing, integration testing, and functional testing
Let's briefly list the meanings of these three tests:
- Unit tests, also known as module testing, are the testing work that verifies the correctness of program modules (the smallest unit of software design). Program units are the smallest testable components of applications. In procedural programming, a unit is a single program, function, process, etc; For object-oriented programming, the smallest unit is the method, which includes methods in the base class (superclass), abstract class, or derived class (subclass) [2] ^ {[2]} [2]
- Integration testing, also known as assembly testing, refers to the testing work of assembling program modules in a one-time or value-added manner to verify the correctness of system interfaces. Integration testing is generally carried out after unit testing and before system testing. Practice shows that although modules can work separately, it cannot be guaranteed that they can also work together when assembled [3] ^ {[3]} [3]
- Functional tests: Functional testing is the verification of various functions of a product, testing each item according to functional test cases, and checking whether the product meets the user's required functions [4] ^ {[4]} [4]
As you can see, each of the three types of testing plays its own role. When writing code, unit testing is usually used, which is simpler, faster, and easier to execute. Therefore, this article only discusses unit testing.
Unit testing using Python
Unit testing in Python is the process of writing a test function that executes a small section of the application to verify the correctness of the code. If there are any issues, an exception will be thrown. For example, the function forty_two() returns 42. The unit testing for this function is as follows:
from app import forty_two
def test_forty_two():
result = forty_two()
assert result == 42
This example is very simple, and it may be more complex in the actual development process. There may also be more than one assert statement.
To execute this unit test, you need to save it as a Python file and then execute it to complete the testing process.
There are two very popular unit testing frameworks in Python, one is Unittest from the standard library, and the other is Pytest. In this article, we will use a hybrid testing solution, and these two packages will use:
- According to the object-oriented programming philosophy, use
unittestpackageTestCaseto build and organize unit tests - Implement assertions using
assertstatements in Python, supplemented by methods inpytest, to enhance the expression ofassertstatements and output more exception information - The execution file containing
pytestperforms the final test, and in this test program, theTestCaseclass in theunittestpackage is fully supported
If you still don't quite understand something, don't worry, just take a look at the example below to understand.
Test Example
Write a program to process integers from 1 to 100: if divisible by 3, output Fizz; Can be divided by 5, output Buzz; Can be divided by 3 and 5 simultaneously, output FizzBuzz; In other cases, print the number. This is a problem that beginners of programming will encounter called "Fizz Buzz".
If you search online, you will find many related items. For example, someone implemented them using the following code.
for i in range(1, 101):
if i % 15 == 0:
print("FizzBuzz")
elif i % 3 == 0:
print("Fizz")
elif i % 5 == 0:
print("Buzz")
else:
print(i)
It is difficult to test the code like the one above. It must be rewritten, like the one below:
def fizzbuzz(i):
if i % 15 == 0:
return "FizzBuzz"
elif i % 3 == 0:
return "Fizz"
elif i % 5 == 0:
return "Buzz"
else:
return i
def main():
for i in range(1, 101):
print(fizzbuzz(i))
if __name__== '__main__':
main()
The main() function is not necessary, but fizzbuzz is indispensable. Then, save the above code as a fizzbuzz.py file, and it can be used as a separate module (for module issues, please refer to the "Python University Practical Tutorial").
Then create a virtual environment (for virtual environment, please refer to the article "Python Virtual Environment") and install pytest in the virtual environment.
(venv) $ pip install pytest
To test whether function fizzbuzz() works properly, the basic idea is to "control variables". For example, first provide 3、6、9 and other numbers that can be divided by 3 to this function, and test whether it returns Fizz.
Following this approach, create a new file and write the following code. Note that the class TestFizzBuzz created here inherits unittest.TestCase and treats it as a module from the previously created fizzbuzz.py, introducing the created function fizzbuzz.
import unittest
from fizzbuzz import fizzbuzz
class TestFizzBuzz(unittest.TestCase):
def test_fizz(self):
for i in [3, 6, 9, 18]:
print('testing', i)
assert fizzbuzz(i) == 'Fizz'
This program is saved as a file named test_fizzbuzz.py and located in the same directory as fizzbuzz.py. Then, enter pytest on the terminal:
(venv) $ pytest
========================== test session starts ===========================
platform darwin -- Python 3.8.6, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /Users/miguel/testing
collected 1 items
test_fizzbuzz.py . [100%]
=========================== 1 passed in 0.03s ============================
The pytest command automatically detects unit tests. Generally, Python files named according to the * test_[something].py or [something]_test.py * pattern will be considered unit tests. In addition, pytest will also search for files with this naming pattern in subdirectories.
If it is a large project, unit testing should be conducted in an orderly manner. A common method is to place the .py file used for testing in a directory called tests, so as to separate it from the application code.
For example, for the above application fizzbuzz.py, if you want to test the performance of encountering numbers that cannot be divided by 3, you can add a number 4 to the list of test_fizzbuzz.py and then run pytest.
(venv) $ pytest
========================== test session starts ===========================
platform darwin -- Python 3.8.6, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /Users/miguel/testing
collected 1 item
test_fizzbuzz.py F [100%]
================================ FAILURES ================================
_________________________TestFizzBuzz.test_fizz _________________________
self = <test_fizzbuzz.TestFizzBuzz testMethod=test_fizz>
def test_fizz(self):
for i in [3, 4, 6, 9, 18]:
print('testing', i)
> assert fizzbuzz(i) == 'Fizz'
E AssertionError: assert 4 == 'Fizz'
E + where 4 = fizzbuzz(4)
test_fizzbuzz.py:9: AssertionError
-------------------------- Captured stdout call --------------------------
testing 3
testing 4
======================== short test summary info =========================
FAILED test_fizzbuzz.py::TestFizzBuzz::test_fizz - AssertionError: asse...
=========================== 1 failed in 0.13s ===========================
Note that once encountering numbers that do not meet the criteria, the testing program will assert and stop. In order to accurately locate the location of the failure, pytest will display the source code and mark the assertion location and actual execution results. In addition, it will automatically output the test content. For example, as shown in the test report above, tests were conducted on the numbers 3 and 4, and test 4 failed. After the test failed, it will return to the initial conditions of the test.
The testing of Fizz has been completed, and similarly, further testing of Buzz and FizzBuzz can be added:
import unittest
from fizzbuzz import fizzbuzz
class TestFizzBuzz(unittest.TestCase):
def test_fizz(self):
for i in [3, 6, 9, 18]:
print('testing', i)
assert fizzbuzz(i) == 'Fizz'
def test_buzz(self):
for i in [5, 10, 50]:
print('testing', i)
assert fizzbuzz(i) == 'Buzz'
def test_fizzbuzz(self):
for i in [15, 30, 75]:
print('testing', i)
assert fizzbuzz(i) == 'FizzBuzz'
Now running pytest again will display three unit tests that have all passed:
(venv) $ pytest
========================== test session starts ===========================
platform darwin -- Python 3.8.6, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /Users/miguel/testing
collected 3 items
test_fizzbuzz.py ... [100%]
=========================== 3 passed in 0.04s ============================
Test coverage
Are you satisfied with the above three tests?.
Although you must determine how much automated testing is needed based on your own experience to ensure that the program will not experience bugs in the future, there is a concept or tool for this: Code coverage It can help developers better implement unit testing.
Install another module: pytest cov]（ https://pytest-cov.readthedocs.io/en/latest/ ）By using it, the code coverage of the test can be detected.
(venv) $ pip install pytest-cov
Execute the command pytest --cov=fizzbuzz, run the unit test, and note that in the parameter list on the command line, it is declared to enable code coverage tracking for the fizzbuzz module:
(venv) $ pytest --cov=fizzbuzz
========================== test session starts ===========================
platform darwin -- Python 3.8.6, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /Users/miguel/testing
plugins: cov-2.11.1
collected 3 items
test_fizzbuzz.py ... [100%]
---------- coverage: platform darwin, python 3.8.6-final-0 -----------
Name Stmts Miss Cover
---------------------------------
fizzbuzz.py 13 4 69%
---------------------------------
TOTAL 13 4 69%
=========================== 3 passed in 0.07s ============================
Please note that the command line above uses the --cov parameter to set the limit range for code coverage, which is the fizzbuzz module. If this parameter is not set, a lot of content will be output in the final test report, including but not limited to testing Python standard libraries, third-party libraries, etc., presenting a dazzling report.
Through this report, we can see that the three unit tests covered 69% of fizzbuzz.py code, while the remaining 31% were not covered. It is also necessary to know what the untested code is. The method is to add a command line parameter.
pytest-cov provides multiple formats of final reports. As executed below, adding --cov-report=term-missing will add a column of Missing to the final report, which will display uncovered code lines.
(venv) $ pytest --cov=fizzbuzz --cov-report=term-missing
========================== test session starts ===========================
platform darwin -- Python 3.8.6, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /Users/miguel/testing
plugins: cov-2.11.1
collected 3 items
test_fizzbuzz.py ... [100%]
---------- coverage: platform darwin, python 3.8.6-final-0 -----------
Name Stmts Miss Cover Missing
-------------------------------------------
fizzbuzz.py 13 4 69% 9, 13-14, 18
-------------------------------------------
TOTAL 13 4 69%
=========================== 3 passed in 0.07s ============================
The report shows that the 13th and 14th lines in fizzbuzz.py were not covered by unit testing, and the two lines in main() actually have nothing to do with the part we really want to test. It is only natural that they are not covered.
In addition, there is the 18th line, which is the last line of fizzbuzz.py, which returns to take a look at the source program. Its function is only to execute this script, and it is not a test object.
However, the 9th line mentioned in the report, which has not yet been covered, is a line in the fizzbuzz() function. Although we are testing this function, it seems that there are still omissions. However, the 9th line is the last line of the function, which returns the number after determining that the input number cannot be divided by 3 or 5. Therefore, it is necessary to add a unit test specifically to check for numbers that are not Fizz, Buzz, or FizzBuzz.
Compared to the source file fizzbuzz.py, the unit test above did not test the if conditional statement. If you want to overwrite it, you need to add --cov-branch in the command line:
(venv) $ pytest --cov=fizzbuzz --cov-report=term-missing --cov-branch
========================== test session starts ===========================
platform darwin -- Python 3.8.6, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /Users/miguel/testing
plugins: cov-2.11.1
collected 3 items
test_fizzbuzz.py ... [100%]
---------- coverage: platform darwin, python 3.8.6-final-0 -----------
Name Stmts Miss Branch BrPart Cover Missing
---------------------------------------------------------
fizzbuzz.py 13 4 10 2 65% 6->9, 9, 13-14, 17->18, 18
---------------------------------------------------------
TOTAL 13 4 10 2 65%
=========================== 3 passed in 0.07s ============================
Now, the test coverage has decreased to 65%, and not only the 9th, 13th, 14th, and 18th rows are displayed under column Missing, but also the rows containing only Boolean conditional statements have been added.
As mentioned earlier, we need to add another unit test to test numbers that cannot be divided by 3 or 5. Let's use this unit test to cover line 9:
import unittest
from fizzbuzz import fizzbuzz
class TestFizzBuzz(unittest.TestCase):
def test_fizz(self):
for i in [3, 6, 9, 18]:
print('testing', i)
assert fizzbuzz(i) == 'Fizz'
def test_buzz(self):
for i in [5, 10, 50]:
print('testing', i)
assert fizzbuzz(i) == 'Buzz'
def test_fizzbuzz(self):
for i in [15, 30, 75]:
print('testing', i)
assert fizzbuzz(i) == 'FizzBuzz'
def test_number(self):
for i in [2, 4, 88]:
print('testing', i)
assert fizzbuzz(i) == i
Execute the pytest command again, pay attention to the following command line parameters, and observe the output test report to see if it has improved test coverage.
(venv) $ pytest --cov=fizzbuzz --cov-report=term-missing --cov-branch
========================== test session starts ===========================
platform darwin -- Python 3.8.6, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /Users/miguel/testing
plugins: cov-2.11.1
collected 4 items
test_fizzbuzz.py .... [100%]
---------- coverage: platform darwin, python 3.8.6-final-0 -----------
Name Stmts Miss Branch BrPart Cover Missing
---------------------------------------------------------
fizzbuzz.py 13 3 10 1 74% 13-14, 17->18, 18
---------------------------------------------------------
TOTAL 13 3 10 1 74%
=========================== 4 passed in 0.08s ============================
It looks good, with a coverage rate of 74%, especially covering all lines of the fizzbuzz() function, which is the core of the program.
However, the report still shows that lines 13, 14, and 18 are not covered, and the conditional sentence in line 17 is only partially covered.
Let's first look at lines 17 and 18. Based on our experience, these two lines are definitely safe and do not require testing at all. However, the report above shows that it is not satisfactory - if you have this feeling, the direct approach is to add a comment after the line that does not require testing: # pragma: no cover, which will be recognized by unit testing (as shown in the code below).
def fizzbuzz(i):
if i % 15 == 0:
return "FizzBuzz"
elif i % 3 == 0:
return "Fizz"
elif i % 5 == 0:
return "Buzz"
else:
return i
def main():
for i in range(1, 101):
print(fizzbuzz(i))
if __name__== '__main__': # pragma: no cover
main()
Note that you only need to add comments on line 17, as the unit test program will capture the "exception" on this line and scope this code block, so there is no need to add comments repeatedly on line 18.
Run the test again:
(venv) $ pytest --cov=fizzbuzz --cov-report=term-missing --cov-branch
========================== test session starts ===========================
platform darwin -- Python 3.8.6, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /Users/miguel/testing
plugins: cov-2.11.1
collected 4 items
test_fizzbuzz.py .... [100%]
---------- coverage: platform darwin, python 3.8.6-final-0 -----------
Name Stmts Miss Branch BrPart Cover Missing
---------------------------------------------------------
fizzbuzz.py 11 2 8 0 79% 13-14
---------------------------------------------------------
TOTAL 11 2 8 0 79%
=========================== 4 passed in 0.07s ============================
This report looks much fresher.
Should lines 13 and 14 also be marked as exempt from coverage? You decide.
Conclusion
Unit testing is an essential part of development, and it's best not to push it all to testers.