David Amos: Never Modify Inputs Without Permission

Have you ever loaned something to someone only to have it returned to you in a different state?

I saw this exact situation play out this week at work. A user reported that an SDK method, which involved a sequence of API calls, was failing after the first call. Investigation revealed that the HTTP client used by the SDK was silently modifying a headers object shared between API calls.

The software equivalent of borrowing your friend&aposs car and casually returning it with a different colored hood.

Tisk tisk.

The workaround, at least until the bug is fixed in the HTTP client, is to defensively copy the headers object before passing it to the client. But the whole incident got me thinking about mutability and code etiquette.

How you handle input matters.

Mutability is a feature, not a bug.

For the HTTP client, mutating the header object means fewer memory allocations and improved performance. That&aposs a big benefit of mutability. But it can carry significant risk.

For me, the bottom line is:

That&aposs the mistake the HTTP client made.

Outputs should be clearly documented, including implicit ones like side effects. Julia&aposs convention of appending the ! symbol to the names of functions that mutate their arguments strikes me as a useful pattern for loudly exposing this behavior. The ! alerts users to side effects every time they use the function.

There&aposs a good case to be made for never mutating objects, though. Enforcing immutability solves a lot of headaches because it:

• Eliminates entire classes of bugs and foot guns.
• Increases confidence in code correctness, especially in complex architectures where mutability invites action at a distance.
• Makes concurrency easier to deal with.

Whether or not you should strictly maintain immutability in your code is a question you&aposll have to answer for yourself or with your team. Without a doubt, though, one situation where immutability helps is dealing with untrusted code.

The key is to get defensive.

The incident with the HTTP client was timely for me, in a way.

I&aposve been reading Grokking Simplicity by Eric Normand. The book looks at how functional programming is used in practice, largely avoiding the theory by focusing on how functional concepts can improve real-world systems. Not two weeks prior, I read the section on defensive copying.

It&aposs exactly how the SDK team handled the issue with the HTTP client:

Let me clarify what I mean by untrusted code.

The pessimist in me believes that all code is untrustworthy. But that assumption isn&apost always practical in the day-to-day business of writing code. At a minimum, I think of untrusted code as any code that either:

• Displays bad behavior, such as the HTTP client, or
• Can&apost be verified to comply with your expectations or is too costly to modify so that it does, as is often the case with legacy code.

Defensive copying helps in both cases.

Here&aposs the same problem faced by the SDK team expressed in Python:

>>> headers = {"some_key": "some_value"}
>>> response = http_client.get(
    "/endpoint", headers=headers
)
>>> headers
{&apossome_key&apos: &apossome_value&apos,
 &aposboo!&apos: &aposdid\&apost expect me, did ya?&apos}

The key "boo!" with value "didn&apost expect me, did ya?" unexpectedly appears in the headers dictionary. When headers is used in another request, the unexpected contents cause the request to fail.

To implement defensive copying here, make a copy of headers before passing it to http_client.get():

>>> headers = {"some_key": "some_value"}

>>> from copy import deepcopy
>>> response = http_client.get(
    "/endpoint", headers=deepcopy(headers)
)   #                    ^^^^^^^^^^^^^^^^^
>>> #                   /
>>> # A copy of `headers` is sent to the client

>>> headers
{&apossome_key&apos: &apossome_value&apos}

The contents of headers are unchanged because a copy of headers— an entirely distinct object made with Python&aposs deepcopy() function— was sent to the client instead.

Now you can safely pass a new copy of headers to the next request. The client still mutates the copy, but the original object is protected. That protection comes at the price: increased memory overhead. But in this case, and many others, the improvement in reliability justifies the tradeoff.

Defensive copying doesn&apost just protect objects passed from your code into untrusted code. It also prevents changes to mutable objects passed into your code from untrusted sources.

The headers dictionary, as seen from the perspective of the HTTP client, originates from code that uses the client&aposs API — a classic example of untrusted code. Making a copy of the argument passed to the client before mutating it would have saved everyone some trouble.

Maybe it&aposs just me, but it seems like the considerate thing to do, too.

What To Read Next

Learn how to recognize and remove implicit outputs, like the mutated header object in this week&aposs example, in your own code:

Stop Using Implicit Inputs And Outputs

One simple way to improve testability and reusability.

davidamos.devDavid Amos

Dig Deeper

Learn more about defensive copying and other strategies for enforcing immutability in Eric Normand&aposs book Grokking Simplicity.

Get instant access from Manning*, or buy a print version from Amazon*.

* Affiliate link. See my affiliate disclosure for more information.