Yoong Kang Lim: 3 ways of testing Django views

I tried 3 different ways of testing a Django view. These are my thoughts.

Say we have this Django view that creates a Post that we want to test:

# views.pyfrom.importformsdefcreate_post(request):form=forms.PostForm(request.POSTorNone)ifform.is_valid():post=form.save()messages.info(request,'Post created.')returnredirect(post)returnrender(request,'post/create.html',{'form':form})

How should I write tests for this view?

Let's look at 3 different ways.

Using the Django test client

Perhaps the most straightforward way is to use the Django test client ¹.

fromdjango.testimportClient,TestCaseclassPostViewTestCase(TestCase):deftest_post_creation(self):c=Client()# instantiate the Django test clientresponse=c.post('/post/create',{'title':'Some title','body':'Some text'})self.assertEqual(response.redirect_chain,[('/post/1/',302)])self.assertContains(response,'Post created.')self.assertContains(response,'Some title')

Pros:

• Straightforward to write
• Easy to understand just by reading the code of the test
• Reasonably representative of what actually happens.

Cons:

• It hits all the code in the view, including writing to the database, and also hits the middleware. Can be slow.
• If the test fails, it might not be obvious why. For example, is it the form, the HTML, the model, the messages framework, or is the response simply formed incorrectly? We only know "something" went wrong.
• You need to setup the whole environment, sometimes including creating data in the database for the test to work (for example, list views or detail views)

Using Selenium

We could also use Selenium with StaticLiveServerTestCase² (requires installing Selenium).

fromdjango.testimportStaticLiveServerTestCasefromselenium.webdriver.firefox.webdriverimportWebDriverclassPostSeleniumTest(StaticLiveServerTestCase):defsetUp(self):self.browser=WebDriver()self.browser.implicitly_wait(10)deftearDown(self):self.browser.quit()deftest_post_creation(self)# Go to Post creation form and create postself.browser.get('/post/create/')title_input=self.browser.find_element_by_name('title')title_input.send_keys('Some title')body_input=self.browser.find_element_by_name('body')body_input.send_keys('Some text')self.browser.find_element_by_xpath('//input[@value="Create post"]').click()# Finally, the assertionsmessages_text=self.browser.find_element_by_id('id_messages').textself.assertIn('Post created.',messages_text)post_text=self.browser.find_element_by_id('id_post').textself.assertIn('Some title',post_text)self.assertIn('Some text',post_text)

Pros:

• Fully tests everything, including problems with HTML
• Accounts for Javascript triggered events (e.g Ajax)
• Most representative of what actually happens
• Can be used to test business-critical workflows (such as getting a prospective customer to sign up and pay)

Cons:

• SLOW!!!!!
• Very complicated to setup (installation, etc)
• Navigation/assertions are painful to write, and you tend to write very long tests (libraries like splinter can help ³)
• Very brittle to changes. A very slight change such as changing the HTML id of an element will cause the assertions to fail, even if it's actually working correctly
• It's even worse at telling you what went wrong if the test fails. It will require you to re-run the test and watch the browser live
• Sometimes, tests can fail for irrelevant reasons, like an Ajax request taking too long to complete before an assertion

Unit tests

In Django's design philosophies page⁴, it says this (emphasis mine):

Use request objects

Views should have access to a request object – an object that stores metadata about the current request. The object should be passed directly to a view function, rather than the view function having to access the request data from a global variable. This makes it light, clean and easy to test views by passing in “fake” request objects.

Nice! That sounds like it's a good thing to use for unit tests.

"Fake" objects are usually called "Test Doubles" -- they're fake objects that respond to the same interface as the real object, but returns fixed values for the purposes of the test (instead of, say, performing an expensive calculation).

We could create a class like this ourselves, but Django provides a utility to do that via the RequestFactory⁵. We can use it to pass it in the views like this:

fromdjango.testimportRequestFactoryfrom.viewsimportsome_viewrequest=RequestFactory().get('/some_url/)response=some_view(request)# just pass it in the view

RequestFactory doesn't handle the middleware for you (and it shouldn't), and you'll have to handle that yourself. So if you have code that writes to the request session, you'll need to mock that out.

Unit tests are different from integration tests (like the one using the test client) in that unit tests are concerned with testing one thing -- in this case, the view.

Things a unit test should do

I think these are the considerations for unit testing the view:

• We should test for the expected response (e.g, status code, content) from a view, given a request
• If the view is supposed to cause a side effect in the world in some fashion (e.g, writing to the database, altering the session, etc.), we want to actually test that the view calls the command(s) to do that. But we do NOT want these commands to run -- only that the view calls them. We typically do this using mocks ⁶.
• Use test stubs for outgoing query⁷ method calls. Stubbing means using a fake object to return known values. We are already stubbing out the request using RequestFactory, but you should also be stubbing out database queries if you have them.

We don't care about anything else. In other words, our unit test should be isolated.

You might be wondering how we know if the side effect actually happened, i.e. how do we know the Post gets created?

We want to test that, but that's not the responsibility of the view's test. That responsibility relies on tests of the thing that the view calls instead.

For example, if we rely on the view to call the save() method in a ModelForm to create that side effect, we should make sure that the view calls it. But, it is the ModelForm tests that are responsible for ensuring that the object gets created.

However, Django already has extensive tests for ModelForms, so we don't have to write tests for them. We can just rely on the framework to work as expected. The caveat for this is you have custom clean() or clean_FOO() methods, which you should be testing. Also, remember that a ModelForm will call the model's full_clean() method, so if you have overridden any of the methods it calls, you should test that too. And also if you have custom validators.

That was a lot of words. Let's look at a concrete example:

Unit test example

fromdjango.testimportTestCase,RequestFactory,mockfrompostimportviews,modelsclassPostTestCase(TestCase):@mock.patch('post.views.messages')@mock.patch('post.views.forms.PostForm)deftest_create_view(self,form_class,messages):request=RequestFactory().post('/post/create/',{'title':'Some title','body':'Some text',})form=form_class.return_valueform.is_valid.return_value=Trueresponse=views.create_post(request)self.assertEqual(response.status_code,302)self.assertEqual(response.url,'/post/1/')form.save.assert_called()# test that save() is calledmessages.info.assert_called_with(request,'Post created.')

This test will run a lot faster than the other two tests, since we don't touch middleware or the database.

It also feels like we have mocked half of the universe to get this test to run. The result is, sometimes unit tests will lie to you. For example, notice that I have this line in the test:

form.save.assert_called()# test that save() is called

If for some reason, the view gets changed to use this code:

post=form.save(commit=False)#... never calls post.save()

The test would still pass, and that's really bad.

To fix, this, you'll need to ensure that you're always calling the method with arguments that cause the side effect:

form.save.assert_called_with()# test that save() is called with no arguments

Pros:

• Very fast
• Failures tend to indicate exactly what went wrong, at the appropriate level of granularity
• Relatively easy to understand the tests (since they test only one thing)
• Tests won't fail due to unrelated reasons. If it fails, it's because the problem is within the view
• If you find unit tests difficult to write, it's an indicator that you have too much coupling and some refactoring is needed

Cons:

• Seems quite far removed from what actually happens in real life
• With too much mocking, we can end up with tests that basically just mirror the implementation
• Unit tests can and will lie to you if you don't mock properly
• You'll need to write a lot more tests

Discussion

The test client is the most straightforward way to test views, and it's reasonably reliable. It can get slow if there are many database operations.

Selenium tests are very slow, and can be brittle, but it will catch all kinds of problems, including HTML elements not displaying correctly.

Unit tests are fast, but mocking can be challenging, and sometimes mocking will cause the tests to lie to you. But if you find mocking challenging, or if you find your unit tests are difficult to write, that can serve as an indicator that your code is tightly coupled and should be refactored.

Which one should we be using? My personal opinion is to use the Django test client for views that have relatively straightforward database operations.

If we have many corner cases in the view, we should also use the Django test client, but only in one or two cases (a success scenario and a fail scenario). We should use unit tests to test all the possible corner cases that we're interested in, and not using the test client multiple times.

An example of this is if we want to test the field errors on a form we submit that relies on a database query in its clean() method. We don't want to be using the test client to post to the view over and over to test each combination of field errors! Just write unit tests for the form and the view.

Selenium tests can be useful if the workflow of the thing you're testing is business critical, like the flow of customers signing up.

Resources

Testing is a deep topic. Let me leave you with a few resources I personally found helpful (mainly talks).

Sandi Metz from the Ruby community gave a talk called The Magic Tricks of Testing which is the basis of what I wrote here on what assertions a unit test should have, including command and query messages. You should watch the whole video, Sandi Metz is a wonderful speaker.

Gary Bernhardt gave a talk called Fast Test, Slow Test that discusses the speed of unit tests, and which tests are really integration tests disguised as unit tests.

Harry Percival wrote a book called Obey the Testing Goat which is a great resource for learning TDD in Django specifically. It's available for free online.

Casey Kinsey also talked about writing fast tests in his talk Writing Fast and Efficient Unit Tests for Django.