You know that person who is always insisting that you use the right capitalisation for your company, you have to use the official font, it’s really important whether you use a dash or a space and you absolutely must use the right terminology in all public communication, etc, etc? Aren’t they annoying? Don’t they seem to be ridiculously focused on silly details that absolutely don’t matter?
I used to feel that way too. I mean I generally humoured them because it always feels like other peoples’ important details don’t matter and that’s usually a sign that you don’t understand their job, but I didn’t really believe that it mattered.
Then I made Hypothesis and now I’m that person.
There’s a long list of guidelines on how I communicate Hypothesis that I have literally never communicated to anyone so I really have no right to get even slightly annoyed when people don’t follow it. To be honest, I have no real right to get annoyed when people don’t follow it even if they have read this post. So consider the intent of this post more “here is what I do, I would appreciate if you do the same when talking about Hypothesis and it will annoy me slightly if you don’t but you are entirely welcome to do your own thing and I appreciate you talking about Hypothesis however you do it”.
Without further ado, here are the Hypothesis brand guidelines.
The little things
- Hypothesis is a testing library, not a testing framework. This is actually important, because a thing that people never seem to realise (possibly because I don’t communicate it clearly enough, but it does say so in the documentation) is that Hypothesis does not have its own test runner, it just uses your normal test runners.
- I try not to use the phrase “Hypothesis tests” (I slip up on this all the time when speaking) because that’s a statistical concept. I generally use “Tests using Hypothesis”. It’s more awkward but less ambiguous.
- Hypothesis isn’t really a Quickcheck. I describe it as “Inspired by Quickcheck” rather than “Based on Quickcheck” or “A Quickcheck port”. It started out life as a Quickcheck port, but modern Hypothesis is a very different beast, both internally and stylistically.
The big thing
All of the classic Quickcheck examples are terrible. Please don’t use them. Pretty please?
In particular I hate the reversing a list example. It’s toy, there’s no easy way to get it wrong, and it’s doing this style of testing a great injustice by failing to showcase all the genuinely nasty edge cases it can find.
In general I try never to show an example using Hypothesis that does not expose a real bug that I did not deliberately introduce. Usually producing this it is enough to write a medium complexity example which you know can be well tested with Hypothesis, then add Hypothesis based tests. You can write it TDD if you like as long as you don’t use Hypothesis to do so.
The Tone
The tone I generally try to go for is “Computers are terrible and you can never devote enough time to testing, so Hypothesis is a tool you can add to your arsenal to make the time you have more effective”.
Disclaimers and safety warnings
- Hypothesis will not find all your bugs.
- Hypothesis will not replace all your example based tests.
- Hypothesis isn’t magic.
- Hypothesis will not grant you the ability to write correct code, only help you understand ways in which your code might be incorrect.
The Rest
There are probably a bunch of things I’ve forgotten on this, and I will update the list as I think of them.