Holger Peters: Why I use py.test

Python's Unittest

To grasp what py.test does it is worth to take a look at Python's default module for unit tests which comes with the standard library: unittest. Unittest is an xUnit, which means it is a descendent of the original SUnit, a unit testing frame work for Smalltalk. Smalltalk, like Java, is a language where methods are the only kind of functions available. So for writing a test, one needed to write a test class, and add each test as a method of that class. These test methods in Smalltalk kind of looked like this:

ExampleSetTest>>setUpfull:=Setwith:5with:6ExampleSetTest>>testIncludesselfassert: (fullincludes:5).selfassert: (fullincludes:6)

Or rather, as in Smalltalk, all development takes place in a method browser, a screenshot give you a better impression of the SUnit experience:

In Smalltalk, editing takes place in a central IDE with a class and method browser.

Python unittest looks kind of similar to this, with the difference that class and method definition appear in the same file (it is the port of a port after all):

classTestExampleSet(unittest.TestCase):defsetUp(self):self.full={5,6}deftest_values_in_full(self):self.assertIn(5,self.full)self.assertIn(6,self.full)

The setUp method is what we called the test fixture, it is a method that is called right before executing all the test_* methods. In our case, this is initializing a set. This method is used for common initializations that are needed in every test. Now we just need to run the tests with:

python -m unittest

It seems a bit weird to use classes for grouping test cases, this is more like what I would solve with namespaces in Python. Again, the reason why unittest organizes tests in classes is: because Smalltalk did.

Nevertheless, the unittest module is a proven tool, that generations of programmers have written unit tests with. Let's take a moment to acknowledge that it is a method that is well-respected and it is totally fine to write tests that way.

It just is not the only way.

pytest

Pytest is a test-runner, an extra module to run unittests as easy as:

py.test

As such, it will run your unittest style unit tests like the one we wrote before (which means that you can pretty easily switch to pytest as a runner for an existing project, without having to rewrite your old unit tests). But: it will also run simple functions that start with test_*:

deftest_values_in_full():assert5in{5,6}assert6in{5,6}

If we want to continue using test fixtures, we can use the pytest.fixture decorator:

importpytest@pytest.fixturedeffull():return{5,6}deftest_values_in_full(full):assert5infullassert6infull

Comparing this to the unittest tests this differs mainly in two aspects, the use of fixtures and of assertions.

Instead of manipulating self of a test object, we just write a simple test function and use a fixture that we have declared with a decorator. pytest's fixture system is composable (unlike the more rigid setUp method). We can combine several, different fixtures. Suppose you have one fixture to secure a database connection, and one fixture to obtain a temporary directory. In pytest you can use them as needed in every test individually.

We use plain Python assert statements instead of self.assert* methods. Pytest will inspect them to come up with a readable error message. I cannot emphasize this enough. Instead of self.assertListEqual(a, b), we can just type assert a == b, and assert foo() replaces self.assertTrue(foo()).

Pytest has accumulated lot's of helpful features for getting the best from testing. Some of which are:

deftest_foo():x=1b="a"assertx==len(b)assert2==len(b)# <- fail

% py.test -l test_f.py
================== test session starts ===================
platform darwin -- Python 3.4.3, pytest-2.8.2, py-1.4.30, pluggy-0.3.1
collected 1 items

test_f.py F

======================== FAILURES ========================
________________________ test_foo ________________________

    def test_foo():
        x = 1
        b = "a"
        assert x == len(b)
h       assert 2 == len(b)
E       assert 2 == 1
E        +  where 1 = len('a')

b          = 'a'
x          = 1

test_f.py:7: AssertionError
================ 1 failed in 0.01 seconds ================

Plugins

What is also great about pytest are the plugins available for it.

pip install pytest-xdist
py.test -n 4 # for 4 cores

pip install pytest-cov
py.test --cov-report html --cov myproject

py.test --cov-report term --cov myproject