Real Python: Python and TOML: New Best Friends

TOML—Tom’s Obvious Minimal Language—is a reasonably new configuration file format that the Python community has embraced over the last couple of years. TOML plays an essential part in the Python ecosystem. Many of your favorite tools rely on TOML for configuration, and you’ll use pyproject.toml when you build and distribute your own packages.

In this tutorial, you’ll learn more about TOML and how you can use it. In particular, you’ll:

• Learn and understand the syntax of TOML
• Use tomli and tomllib to parse TOML documents
• Use tomli_w to write data structures as TOML
• Use tomlkit when you need more control over your TOML files

A new module for TOML parsing is being added to Python’s standard library in Python 3.11. Later in this tutorial, you’ll learn how to use this new module. If you want to know more about why tomllib was added to Python, then have a look at the companion tutorial, Python 3.11 Preview: TOML and tomllib.

Use TOML as a Configuration Format

TOML is short for Tom’s Obvious Minimal Language and is humbly named after its creator, Tom Preston-Werner. It was designed expressly to be a configuration file format that should be “easy to parse into data structures in a wide variety of languages” (Source).

In this section, you’ll start thinking about configuration files and look at what TOML brings to the table.

Configurations and Configuration Files

A configuration is an important part of almost any application or system. It’ll allow you to change settings or behavior without changing the source code. Sometimes you’ll use a configuration to specify information needed to connect to another service like a database or cloud storage. Other times you’ll use configuration settings to allow your users to customize their experience with your project.

Using a configuration file for your project is a good way to separate your code from its settings. It also encourages you to be conscious about which parts of your system are genuinely configurable, giving you a tool to name magic values in your source code. For now, consider this configuration file for a hypothetical tic-tac-toe game:

player_x_color=blueplayer_o_color=greenboard_size=3server_url=https://tictactoe.example.com/

You could potentially code this directly in your source code. However, by moving the settings into a separate file, you achieve a few things:

• You give explicit names to values.
• You provide these values more visibility.
• You make it simpler to change the values.

Look more closely at your hypothetical configuration file. Those values are conceptually different. The colors are values that your framework probably supports changing. In other words, if you replaced blue with red, that would be honored without any special handling in your code. You could even consider if it’s worth exposing this configuration to your end users through your front end.

However, the board size may or may not be configurable. A tic-tac-toe game is played on a three-by-three grid. It’s not certain that your logic would still work for other board sizes. It may still make sense to keep the value in your configuration file, both to give a name to the value and to make it visible.

Finally, the project URL is usually essential when deploying your application. It’s not something that a typical user will change, but a power user may want to redeploy your game to a different server.

To be more explicit about these different use cases, you may want to add some organization to your configuration. One popular option is to separate your configuration into additional files, each dealing with a different concern. Another option is to group your configuration values somehow. For example, you can organize your hypothetical configuration file as follows:

[user]player_x_color=blueplayer_o_color=green[constant]board_size=3[server]url=https://tictactoe.example.com

The organization of the file makes the role of each configuration item clearer. You can also add comments to the configuration file with instructions to anyone thinking about making changes to it.

There are many ways for you to specify a configuration. Windows has traditionally used INI files, which resemble your configuration file from above. Unix systems have also relied on plain-text, human-readable configuration files, although the actual format varies between different services.

Over time, more and more applications have come to use well-defined formats like XML, JSON, or YAML for their configuration needs. These formats were designed as data interchange or serialization formats, usually meant for computer communication.

On the other hand, configuration files are often written or edited by humans. Many developers have gotten frustrated with JSON’s strict comma rules when updating their Visual Studio Code settings or with YAML’s nested indentations when setting up a cloud service. Despite their ubiquity, these file formats aren’t the easiest to write by hand.

TOML: Tom’s Obvious Minimal Language