Anthony Shaw joins Brian to discuss flaky tests and flaky test suites.
- What are flaky tests?
- Is it the same as fragile tests?
- Why are they bad?
- How do we deal with them?
- What causes flakiness?
- How can we fix them?
- How can we avoid them?
- Proactively rooting out flakiness
- Test design
- GUI tests
- Sharing solutions
Special Guest: Anthony Shaw.
Sponsored By:
Support Test and Code
Links:
<p>Anthony Shaw joins Brian to discuss flaky tests and flaky test suites.</p>
<ul>
<li>What are flaky tests?</li>
<li>Is it the same as fragile tests?</li>
<li>Why are they bad?</li>
<li>How do we deal with them?</li>
<li>What causes flakiness?</li>
<li>How can we fix them?</li>
<li>How can we avoid them?</li>
<li>Proactively rooting out flakiness</li>
<li>Test design</li>
<li>GUI tests</li>
<li>Sharing solutions</li>
</ul><p>Special Guest: Anthony Shaw.</p><p>Sponsored By:</p><ul><li><a rel="nofollow" href="https://do.co/testandcode">DigitalOcean</a>: <a rel="nofollow" href="https://do.co/testandcode">Get started with a free $100 credit toward your first project on DigitalOcean and experience everything the platform has to offer, such as: cloud firewalls, real-time monitoring and alerts, global datacenters, object storage, and the best support anywhere. Claim your credit today at: do.co/testandcode</a></li></ul><p><a rel="payment" href="https://www.patreon.com/testpodcast">Support Test and Code</a></p><p>Links:</p><ul><li><a title="Dropbox article on flaky tests" rel="nofollow" href="https://blogs.dropbox.com/tech/2018/05/how-were-winning-the-battle-against-flaky-tests/">Dropbox article on flaky tests</a></li><li><a title="Microsoft article on flaky tests" rel="nofollow" href="https://blogs.msdn.microsoft.com/bharry/2017/06/28/testing-in-a-cloud-delivery-cadence/">Microsoft article on flaky tests</a></li><li><a title="pytest-rerunfailures: a py.test plugin that re-runs failed tests up to -n times to eliminate flakey failures" rel="nofollow" href="https://github.com/pytest-dev/pytest-rerunfailures">pytest-rerunfailures: a py.test plugin that re-runs failed tests up to -n times to eliminate flakey failures</a></li><li><a title="pytest-randomly: Pytest plugin to randomly order tests and control random.seed" rel="nofollow" href="https://github.com/pytest-dev/pytest-randomly">pytest-randomly: Pytest plugin to randomly order tests and control random.seed</a></li><li><a title="pytest-random-order: pytest plugin to randomise the order of tests with some control over the randomness" rel="nofollow" href="https://github.com/jbasko/pytest-random-order">pytest-random-order: pytest plugin to randomise the order of tests with some control over the randomness</a></li><li><a title="math.isclose()" rel="nofollow" href="https://docs.python.org/3/library/math.html#math.isclose">math.isclose()</a></li><li><a title="numpy.isclose()" rel="nofollow" href="https://docs.scipy.org/doc/numpy-1.15.0/reference/generated/numpy.isclose.html">numpy.isclose()</a></li><li><a title="pytest.approx()" rel="nofollow" href="https://docs.pytest.org/en/latest/reference.html#pytest-approx">pytest.approx()</a> — approx</li><li><a title="Anthony's testing article on RealPython" rel="nofollow" href="https://realpython.com/python-testing/">Anthony's testing article on RealPython</a></li><li><a title="Ghost Inspector" rel="nofollow" href="https://ghostinspector.com/">Ghost Inspector</a></li><li><a title="wily: A Python application for tracking, reporting on timing and complexity in tests" rel="nofollow" href="https://github.com/tonybaloney/wily">wily: A Python application for tracking, reporting on timing and complexity in tests</a></li></ul>