Despite being so useful, external APIs can be a pain to test. When you hit an actual API, your tests are at the mercy of the external server, which can result in the following pain points:
- The request-response cycle can take several seconds. That might not seem like much at first, but the time compounds with each test. Imagine calling an API 10, 50, or even 100 times when testing your entire application.
- The API may have rate limits set up.
- The API server may be unreachable. Maybe the server is down for maintenance? Maybe it failed with an error and the development team is working to get it functional again> Do you really want the success of your tests to rely on the health of a server that you don’t control?
Your tests shouldn’t assess whether an API server is running; they should test whether your code is operating as expected.
In the previous tutorial, we introduced the concept of mock objects, demonstrated how you could use them to test code that interacts with external APIs. This tutorial builds on the same topics, but here we walk you through how to actually build a mock server rather than mocking the APIs. With a mock server in place, you can perform end-to-end tests. You can use your application and get actual feedback from the mock server in real time.
When you finish working through the following examples, you will have programmed a basic mock server and two tests – one that uses the real API server and one that uses the mock server. Both tests will access the same service, an API that retrieves a list of users.
NOTE: This tutorial uses Python v3.5.1.
Getting Started
Start by following the First steps section of the previous post. Or grab the code from the repository. Make sure the test passes before moving on:
1234567 |
|
Testing the Mock API
With the set up complete, you can program your mock server. Write a test that describes the behavior:
project/tests/test_mock_server.py
12345678910111213 |
|
Notice that it starts off looking almost identical to the real API test. The URL has changed and is now pointing to an API endpoint on localhost where the mock server will run.
Here is how to create a mock server in Python:
project/tests/test_mock_server.py
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748 |
|
First, create a subclass of BaseHTTPRequestHandler
. This class captures the request and constructs the response to return. Override the do_GET()
function to craft the response for an HTTP GET request. In this case, just return an OK status. Next, write a function to get an available port number for the mock server to use.
The next block of code actually configures the server. Notice how the code instantiates an HTTPServer
instance and passes it a port number and a handler. Next, create a thread, so that the server can be run asynchronously and your main program thread can communicate with it. Make the thread a daemon, which tells the thread to stop when the main program exits. Finally, start the thread to serve the mock server forever (until the tests finish).
Create a test class and move the test function to it. You must add an additional method to ensure that the mock server is launched before any of the tests run. Notice that this new code lives within a special class-level function, setup_class()
.
Run the tests and watch them pass:
1 |
|
Testing a Service that Hits the API
You probably want to call more than one API endpoint in your code. As you design your app, you will likely create service functions to send requests to an API and then process the responses in some way. Maybe you will store the response data in a database. Or you will pass the data to a user interface.
Refactor your code to pull the hardcoded API base URL into a constant. Add this variable to a constants.py file:
project/constants.py
1 |
|
Next, encapsulate the logic to retrieve users from the API into a function. Notice how new URLs can be created by joining a URL path to the base.
project/services.py
123456789101112131415161718 |
|
Move the mock server code from the feature file to a new Python file, so that it can easily be reused. Add conditional logic to the request handler to check which API endpoint the HTTP request is targeting. Beef up the response by adding some simple header information and a basic response payload. The server creation and kick off code can be encapsulated in a convenience method, start_mock_server()
.
project/tests/mocks.py
123456789101112131415161718192021222324252627282930313233343536373839404142 |
|
With your changes to the logic completed, alter the tests to use the new service function. Update the tests to check the increased information that is being passed back from the server.
project/tests/test_real_server.py
12345678910111213 |
|
project/tests/test_mock_server.py
12345678910111213141516171819202122232425 |
|
Notice a new technique being used in the test_mock_server.py* code. The
response = get_users()line is wrapped with a
patch.dict()` function from the mock library.
What does this statement do?
Remember, you moved the requests.get()
function from the feature logic to the get_users()
service function. Internally, get_users()
calls requests.get()
using the USERS_URL
variable. The patch.dict()
function temporarily replaces the value of the USERS_URL
variable. In fact, it does so only within the scope of the with
statement. After that code runs, the USERS_URL
variable is restored to its original value. This code patches the URL to use the mock server address.
Run the tests and watch them pass.
12345678910 |
|
Skipping Tests that Hit the Real API
We began this tutorial describing the merits of testing a mock server instead of a real one, however, your code currently tests both. How do you configure the tests to ignore the real server? The Python ‘unittest’ library provides several functions that allow you to skip tests. You can use the conditional skip function ‘skipIf’ along with an environment variable to toggle the real server tests on and off. In the following example, we pass a tag name that should be ignored:
1 |
|
project/constants.py
123456 |
|
project/tests/test_real_server.py
123456789101112131415161718 |
|
Run the tests and pay attention to how the real server test is ignored:
12345678910 |
|
Next Steps
Now that you have created a mock server to test your external API calls, you can apply this knowledge to your own projects. Build upon the simple tests created here. Expand the functionality of the handler to mimic the behavior of the real API more closely.
Try the following exercises to level up:
- Return a response with a status of HTTP 404 (not found) if a request is sent with an unknown path.
- Return a response with a status of HTTP 405 (method not allowed) if a request is sent with a method that is not allowed (POST, DELETE, UPDATE).
- Return actual user data for a valid request to
/users
. - Write tests to capture those scenarios.
Grab the code from the repo.