Python Bytes: #288 Performance benchmarks for Python 3.11 are amazing

<p><strong>Watch the live stream:</strong></p> <a href='https://www.youtube.com/watch?v=2ZTEEy1_Gkk' style='font-weight: bold;'>Watch on YouTube</a><br> <br> <p><strong>About the show</strong></p> <p>Sponsored by us! Support our work through:</p> <ul> <li>Our <a href="https://training.talkpython.fm/"><strong>courses at Talk Python Training</strong></a></li> <li><a href="https://testandcode.com/"><strong>Test &amp; Code</strong></a> Podcast</li> <li><a href="https://www.patreon.com/pythonbytes"><strong>Patreon Supporters</strong></a></li> </ul> <p><strong>Brian #1:</strong> <a href="https://www.pola.rs/"><strong>Polars: Lightning-fast DataFrame library for Rust and Python</strong></a></p> <ul> <li>Suggested by a several listeners</li> <li>“Polars is a blazingly fast DataFrames library implemented in Rust using <a href="https://arrow.apache.org/docs/format/Columnar.html">Apache Arrow Columnar Format</a> as memory model. <ul> <li>Lazy | eager execution</li> <li>Multi-threaded</li> <li>SIMD (Single Instruction/Multiple Data)</li> <li>Query optimization</li> <li>Powerful expression API</li> <li>Rust | Python | ...”</li> </ul></li> <li>Python API syntax set up to allow parallel and execution while sidestepping GIL issues, for both lazy and eager use cases. From the docs: <a href="https://pola-rs.github.io/polars-book/user-guide/dsl/groupby.html#do-not-kill-the-parallelization">Do not kill parallelization</a></li> <li><p>The syntax is very functional and pipeline-esque:</p> <pre><code>import polars as pl q = ( pl.scan_csv("iris.csv") .filter(pl.col("sepal_length") &gt; 5) .groupby("species") .agg(pl.all().sum()) ) df = q.collect() </code></pre></li> <li><p><a href="https://pola-rs.github.io/polars-book/user-guide/">Polars User Guide</a> is excellent and looks like it’s entirely written with Python examples.</p></li> <li>Includes <a href="https://pola-rs.github.io/polars-book/user-guide/dsl/video_intro.html">a 30 min intro video from PyData Global 2021</a></li> </ul> <p><strong>Michael #2:</strong> <a href="https://lp.jetbrains.com/python-developers-survey-2021/"><strong>PSF Survey is out</strong></a></p> <ul> <li>Have a look, their page summarizes it better than my bullet points will.</li> </ul> <p><strong>Brian #3:</strong> <a href="https://github.com/google/gin-config"><strong>Gin Config: a lightweight configuration framework for Python</strong></a></p> <ul> <li>Found through Vincent D. Warmerdam’s excellent <a href="https://calmcode.io/gin/intro-to-gin.html">intro videos on gin on calmcode.io</a></li> <li>Quickly make parts of your code configurable through a configuration file with the <code>@gin.configurable</code> decorator.</li> <li><p>It’s in interesting take on config files. (Example from Vincent)</p> <pre><code> # simulate.py @gin.configurable def simulate(n_samples): ... # config.py simulate.n_samples = 100 </code></pre></li> <li><p>You can specify:</p> <ul> <li>required settings: <code>def</code> <code>simulate</code>(n_samples=gin.REQUIRED)`</li> <li>blacklisted settings: <code>@gin.configurable(blacklist=["n_samples"])</code></li> <li>external configurations (specify values to functions your code is calling)</li> <li>can also references to other functions: <code>dnn.activation_fn = @tf.nn.tanh</code></li> </ul></li> <li>Documentation suggests that it is especially useful for machine learning.</li> <li>From motivation section: <ul> <li>“Modern ML experiments require configuring a dizzying array of hyperparameters, ranging from small details like learning rates or thresholds all the way to parameters affecting the model architecture.</li> <li>Many choices for representing such configuration (proto buffers, tf.HParams, ParameterContainer, ConfigDict) require that model and experiment parameters are duplicated: at least once in the code where they are defined and used, and again when declaring the set of configurable hyperparameters.</li> <li>Gin provides a lightweight dependency injection driven approach to configuring experiments in a reliable and transparent fashion. It allows functions or classes to be annotated as <code>@gin.configurable</code>, which enables setting their parameters via a simple config file using a clear and powerful syntax. This approach reduces configuration maintenance, while making experiment configuration transparent and easily repeatable.”</li> </ul></li> </ul> <p><strong>Michael #4:</strong> <a href="https://twitter.com/EduardoOrochena/status/1534913062356099079"><strong>Performance benchmarks for Python 3.11 are amazing</strong></a></p> <ul> <li>via Eduardo Orochena</li> <li>Performance may be the biggest feature of all</li> <li>Python 3.11 has <ul> <li>task groups in asyncio</li> <li>fine-grained error locations in tracebacks</li> <li>the self-type to return an instance of their class</li> </ul></li> <li>The "Faster CPython Project" to speed-up the reference implementation. <ul> <li>See my interview with Guido and Mark: <a href="https://talkpython.fm/339"><strong>talkpython.fm/339</strong></a></li> <li>Python 3.11 is 10~60% faster than Python 3.10 according to the official figures</li> <li>And a 1.22x speed-up with their standard benchmark suite.</li> </ul></li> <li>Arriving as stable until October</li> </ul> <p><strong>Extras</strong> </p> <p>Michael:</p> <ul> <li><a href="https://www.python.org/downloads/release/python-3105/"><strong>Python 3.10.5 is available</strong></a> (<a href="https://docs.python.org/release/3.10.5/whatsnew/changelog.html#python-3-10-5-final">changelog</a>)</li> <li><a href="https://www.raycast.com"><strong>Raycast</strong></a> (vs Spotlight) <ul> <li>e.g. CMD+Space =&gt; pypi search: <img src="https://paper-attachments.dropbox.com/s_89E5FC1F5DBBED983E2558E0DE9902BF8E7733F0236C1030506F258E61A5C2AF_1655219401413_pypisearch.png" alt="" /></li> </ul></li> </ul> <p><strong>Joke:</strong> <a href="https://devhumor.com/media/why-wouldn-t-you-choose-a-parrot-for-your-next-application"><strong>Why wouldn't you choose a parrot for your next application</strong></a></p>