Python Bytes: #311 Catching Memory Leaks with ... pytest?

<a href='https://www.youtube.com/watch?v=b1nMpeVY2yM' style='font-weight: bold;'>Watch on YouTube</a><br> <br> <p><strong>About the show</strong></p> <h1>Python Bytes 311</h1> <p>Sponsored by <a href="http://pythonbytes.fm/foundershub2022"><strong>Microsoft for Startups Founders Hub</strong></a>.</p> <p><strong>Connect with the hosts</strong></p> <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy"><strong>@mkennedy@fosstodon.org</strong></a></li> <li>Brian: <a href="https://fosstodon.org/@brianokken"><strong>@brianokken@fosstodon.org</strong></a></li> <li>Special guest: <strong><a href="https://fosstodon.org/@murilocunha">Murilo Cunha</a></strong></li> </ul> <p><strong>Michael #1:</strong> <a href="https://twitter.com/btskinn/status/1592892053364805632"><strong>Latexify</strong></a></p> <ul> <li>We are used to turning beautiful math into programming symbols.</li> <li>For example: <a href="https://amitness.com/2019/08/math-for-programmers/#sigma">amitness.com/2019/08/math-for-programmers/#sigma</a></li> <li>Take this code: def do_math(a, b, c): return (-b + math.sqrt(b ** 2 - 4 * a * c)) / (2 * a)</li> <li>Add <code>@latexify.function</code> decorator</li> <li>display <code>do_math</code> in a notebook</li> <li>Get this latex: <code>\mathrm{do_math}(a, b, c) = \frac{-b + \sqrt{b^{{2}} - {4} a c}}{{2} a}</code></li> <li><p>Which renders as <img src="https://python-bytes-static.nyc3.digitaloceanspaces.com/latex-math.png" alt="" /></p></li> <li><p>I could only get it to install with: <code>pip install git+https://github.com/google/latexify_py</code></p></li> </ul> <p><strong>Brian #2:</strong> <a href="https://pypi.org/project/prefixed/"><strong>prefixed</strong></a></p> <ul> <li>From <strong>Avram Lubkin</strong></li> <li><p>“Prefixed provides an alternative implementation of the built-in <a href="https://docs.python.org/3/library/functions.html#float">float</a> which supports formatted output with <a href="https://en.wikipedia.org/wiki/Metric_prefix">SI (decimal)</a> and <a href="https://en.wikipedia.org/wiki/Binary_prefix">IEC (binary)</a> prefixes.”</p> <pre><code>&gt;&gt;&gt; from prefixed import Float &gt;&gt;&gt; f'{Float(3250):.2h}' '3.25k' &gt;&gt;&gt; '{:.2h}s'.format(Float(.00001534)) '15.34μs' &gt;&gt;&gt; '{:.2k}B'.format(Float(42467328)) '40.50MiB' &gt;&gt;&gt; f'{Float(2048):.2m}B' '2.00KB' </code></pre></li> <li><p>Because <a href="https://prefixed.readthedocs.io/en/stable/api.html#prefixed.Float">prefixed.Float</a> inherits from the built-in <a href="https://docs.python.org/3/library/functions.html#float">float</a>, it behaves exactly the same in most cases.</p></li> <li>When a math operation is performed with another real number type (<a href="https://docs.python.org/3/library/functions.html#float">float</a>, <a href="https://docs.python.org/3/library/functions.html#int">int</a>), the result will be a <a href="https://prefixed.readthedocs.io/en/stable/api.html#prefixed.Float">prefixed.Float</a> instance.</li> <li>also interesting <ul> <li><a href="https://metricviews.uk/2022/11/20/first-new-si-prefixes-for-over-30-years/">First new SI prefixes for over 30 years</a></li> <li><a href="https://en.wikipedia.org/wiki/Metric_prefix#List_of_SI_prefixes">new prefixes also show up here</a></li> </ul></li> - </ul> <p><strong>Murilo #3:</strong> <a href="https://www.getdbt.com/"><strong>dbt</strong></a></p> <ul> <li>Open source tool</li> <li>CLI tool</li> <li>Built with Python 🐍</li> <li>Applies “best practices” to SQL projects</li> <li>Combines git + <code>.sql</code> files + <code>jinja</code></li> <li><a href="https://docs.getdbt.com/docs/supported-data-platforms">Support many data platforms</a></li> <li>Let’s you <ul> <li>Template SQL queries <ul> <li>Including loops</li> </ul></li> <li>Execute DAGs</li> <li>Data validation</li> <li>Easily build docs (data lineage, visualize DAGs, etc.)</li> </ul></li> <li>Now you can also run <a href="https://docs.getdbt.com/docs/build/python-models">Python models</a> <ul> <li>Useful if there’s a convenient python function for your data transformation or some more complex logic (i.e.:fuzzy string matching, machine learning models, etc.)</li> <li><a href="https://docs.getdbt.com/docs/build/python-models#supported-data-platforms">Available for Snowflake, Databricks, BigQuery</a></li> <li>dbt’s coalesce’s announcement https://www.youtube.com/watch?v=rVprdyxcGUo</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href="https://twitter.com/roman_the_right/status/1592938538214912000"><strong>Memray</strong></a> <a href="https://twitter.com/roman_the_right/status/1592938538214912000"><strong>p</strong></a><a href="https://twitter.com/roman_the_right/status/1592938538214912000"><strong>ytest plugin</strong></a></p> <ul> <li><a href="https://github.com/bloomberg/pytest-memray">pytest-memray</a> is the pytest plugin for, well, <a href="https://github.com/bloomberg/memray">memray</a>. :)</li> <li>You can ensure that not too much memory is used with <code>@pytest``**.**``mark``**.**``limit_memory``**(**``"24 MB"``**)**</code> </li> <li>And you get an allocation report with <code>pytest --memray file.py</code></li> <li>But coming soon, we’ll have memory leak checking too. <pre><code>@pytest.mark.check_leaks() def test_foobar(): # Do some stuff and ensure # it does not leak memory pass </code></pre></li> </ul> <p><strong>Brian #5:</strong> <a href="https://textual.textualize.io/blog/2022/11/20/stealing-open-source-code-from-textual/"><strong>Stealing Open Source code from Textual</strong></a></p> <ul> <li>Will McGugan</li> <li>Will reminds us of one of the great benefits of open source code, stealing code <ul> <li>(when allowed by the license, of course)</li> </ul></li> <li>Goes as far as to point out some bits of textual that you might want to lift <ul> <li>looping with indication of when you’ve hit the first or last item</li> <li>a LRUCache with more flexibility than lru_cache</li> <li>a Color class with conversions for css, hex, monochrome, hsl</li> <li>2d geometry</li> </ul></li> </ul> <p><strong>Murilo #6:</strong> <a href="https://github.com/Zac-HD/shed"><strong>Shed</strong></a></p> <ul> <li>Superset of black</li> <li>"<code>shed</code> is the <em>maximally opinionated</em> autoformatting tool. It's <em>all about</em> <a href="https://en.wikipedia.org/wiki/Convention_over_configuration">convention over configuration</a>, and designed to be a single opinionated tool that fully canonicalises my code - formatting, imports, updates, and every other fix I can possibly automate.”</li> <li>Also format code snippets in docstrings, markdown, restructured text</li> <li>No configuration options</li> <li>pre-commit hooks available</li> <li>Bundles together: <ul> <li><code>black</code></li> <li><code>isort</code></li> <li><code>autoflake</code></li> <li><code>pyupgrade</code></li> <li><code>blacken-docs</code></li> </ul></li> </ul> <p><strong>Extras</strong> </p> <p>Brian: </p> <ul> <li><a href="https://pypi.org/project/pytest-check/">pytest-check</a> (version 1.1.3) changes now live <ul> <li>New README, hopefully makes it clear how to use.</li> <li>Use <code>check</code> from <ul> <li><code>from pytest_check import check</code></li> <li>or from the <code>check</code> fixture: <code>def test_foo(check): …</code></li> </ul></li> <li>Either form returns the same object.</li> <li>From that <code>check</code> object, you can <ul> <li>use helper functions like <code>check.equal(a, b)</code>, etc.</li> <li>use it as a context manager, <code>with check: assert a == b</code></li> <li>even grab the <code>raises</code> context manager: <code>with check.raises(Exception): …</code></li> </ul></li> <li>Intended to be backwards compatible <ul> <li>although some old use cases might be deprecated/removed in the future.</li> </ul></li> </ul></li> </ul> <p>Michael:</p> <ul> <li>New YouTube Video: <a href="https://www.youtube.com/watch?v=oNT2Sa_0YJU"><strong>Best Native App for Mastodon is ...</strong></a></li> <li>Nearly 50% of macOS malware <a href="https://www.laptopmag.com/news/nearly-50-of-macos-malware-comes-from-one-app-do-you-have-it-on-your-macbook"><strong>comes from one app</strong></a> — do you have it on your MacBook?</li> <li><a href="https://pretalx.com/pycascades-2023/cfp">PyCascades CfP</a></li> <li>A fresh take on blogging (for Michael): <a href="https://mkennedy.codes"><strong>mkennedy.codes</strong></a> <ul> <li>Based on <a href="https://gohugo.io"><strong>Hugo</strong></a> - which is <em>so</em> good.</li> <li>Hosted on <a href="https://www.netlify.com"><strong>netlify.com</strong></a></li> </ul></li> </ul> <p>Murilo:</p> <ul> <li><a href="https://github.com/halcy/Mastodon.py">mastodon.py</a> - a Python wrapper around <a href="https://github.com/mastodon/mastodon/">Mastodon’s API</a></li> <li>Nice <a href="https://twitter.com/hamelhusain/status/1590956886496419840?s=46&amp;t=QBEo8HahEDTKEhJrsnlcgA">notebook diffs in Github PRs</a> 🚀</li> <li><a href="https://www.youtube.com/watch?v=1FIgj9Q5e6A">flake8 is not on Gitlab anymore</a></li> <li><a href="https://www.youtube.com/watch?v=KjISuZ5o06Q">Who’s gonna win the world cup?</a></li> <li><a href="https://github.com/LeviBorodenko/lancer">lancer</a></li> </ul> <p><strong>Joke:</strong> </p> <ul> <li><a href="https://twitter.com/PR0GRAMMERHUM0R/status/1588939225180905478"><strong>Messing with the algorithm</strong></a> </li> <li>Let’s start this one with <a href="https://despair.com/collections/demotivators">some history</a></li> <li><a href="https://twitter.com/btskinn/status/1593002791978422272">Recusion joke</a></li> </ul>