PyCharm: Python 3.6: A quick look

The new version of Python, version 3.6, is planned to be released on Friday December 16th. Let’s take a quick look at the new features that are included in this release, and play around with it a little.

Python 3.6 has many new cool features, like format strings, a secrets module, file system protocol, and more. If you want to read more about all features, check out python.org.

To show some of what can be done in the new language, let’s create a simple application in modern Python, and then see how we can upgrade it with Python 3.6 features.

Quick disclaimer: this post is not a full list of Python 3.6 features, if you’re interested in that, check out python.org, they have a great summary. This post shows how to actually use some of the new features to improve your programming. And we’ll show you how to use PyCharm with the new Python features.

Now that that’s out of the way, let’s have a look at the application we’ll be upgrading.

The Application

We will make a very simple single-file application, that will take a URL to a file, and download the file while simultaneously calculating the file’s sha256 checksum. If you would like to follow along, check out the GitHub repo. The master branch contains the Python 3.6 code, and the ‘python-3.5’ branch contains the Python 3.5 code.

Now let’s have a look how we can do this in Python 3.5:

import argparse
import asyncio
import hashlib
from urllib.parse import urlsplit

import aiohttp

chunk_size = 1024  # Set to 1 KB chunks


async def download_url(url, destination):
    print('Downloading {}'.format(url))
    file_hash = hashlib.sha256()
    with open(destination, 'wb') as file:
        async with aiohttp.ClientSession() as session:
            async with session.get(url) as response:
                while True:
                    chunk = await response.content.read(chunk_size)
                    if not chunk:
                        break
                    file_hash.update(chunk)
                    file.write(chunk)

    print('Downloaded {}, sha256: {}'.format(destination, file_hash.hexdigest()))


def main():
    # get the URL from the command-line arguments
    parser = argparse.ArgumentParser()
    parser.add_argument('url', metavar='URL', help='The URL to download')
    arguments = parser.parse_args()

    # get the filename from the URL
    url_parts = urlsplit(arguments.url)
    file_name = url_parts.path[url_parts.path.rfind('/') + 1:]

    # start the download async
    loop = asyncio.get_event_loop()
    loop.run_until_complete(download_url(arguments.url, file_name))


if __name__ == '__main__':
    main()

The main parses the argument (the URL) and then starts the asyncio event loop. If you’ve never worked with asyncio before, check out Nick Coghlan’s introduction.

The core of the code is in the download_url function, aiohttp is used to asynchronously download the file in byte-sized chunks (set using the ‘chunk_size’ constant at the top of the file).

As every chunk is received, the sha256sum calculation is updated with the newly received chunk, and that chunk is saved to the disk.

To run the code, first install the dependencies using pip install -r requirements.txt, and then run download.py <URL>. This should result in the following output:

So now let’s see how we can make our application better with Python 3.6 features!

Python 3.6

The first thing we can do is use those fancy new f-strings. Most Python programmers are probably aware that there are several methods in Python to insert data into a string. The classic way would be using the ‘%’ operator, which has been deprecated due to several issues (see PEP 3101). A newer method would be using either string.format class method, or the str.format instance method on a specific string.

In Python 3.6 a new notation is introduced, which combines the variables with the format string, in a way which looks similar to a templating language:

world = "world"
'Hello, {}'.format(world)
# becomes
f'Hello, {world}'

If you’re using PyCharm, it can help you convert these statements: just put your caret in the formatted string, and use Alt+Enter to select the “convert to f-string literal’ intention.

Afterwards these statements will look like this:

print(f'Downloaded {destination}, sha256: {file_hash.hexdigest()}')

Another new feature in Python 3.6 are underscores in numbers, which allows us to add an underscore to the chunk_size constant. This makes it slightly easier to read:

chunk_size = 1_024  # Set to 1 KB chunks

Currently, the download_url function isn’t looking very pretty, a fair amount of boilerplate is mixed in with the business logic. We can’t make it all go away without some serious development, but we can separate some of the flow control away from the business logic using Python 3.6 features. In Python 3.5 and earlier, using a yield statement in an async function was a SyntaxError. In the new Python version asynchronous generators are supported, so we can restructure the while True into a closure which exposes a generator.

async def get_bytes():
    while True:
        chunk = await response.content.read(chunk_size)
        if not chunk:
            return
        yield chunk


# handle the download async
for chunk in get_bytes():
    file_hash.update(chunk)
    file.write(chunk)

There’s one more change that we can make to make our lives easier: we can add an annotation to the file_hash variable. If you were to type file_hash. you currently wouldn’t see any code completion due to the way that hashlib instances the object. We can help PyCharm (and any other static code analysis tool) out by adding a type annotation:

file_hash: _sha256.SHA256Type = hashlib.sha256()

After adding the annotation (and the _sha256 import statement at the top of the document), PyCharm’s code completion works as intended:

In this specific case it isn’t recommended to add this annotation, as the _sha256 module is annotated as a private. However, the way to add annotations is the same for all classes.

A quick note about type annotations: they don’t change anything about how the program works. If you store a number in a variable you annotate as a string, the python interpreter will just ignore your annotation. The annotations are purely for external tools that can analyze your code.

There are more features in Python 3.6, if you’d like to learn about them go to python.org.

PyCharm 2016.3 fully supports Python 3.6, so get the repo and play around. If you don’t have PyCharm 2016.3 yet, have a look and see what’s new.

-PyCharm Team
The Drive to Develop