dynaconf - The dynamic configurator for your Python Project

dynaconf a layered configuration system for Python applications - with strong support for 12-factor applications and extensions for Flask and Django.

Features

• Strict separation of settings from code (following 12-factor applications Guide).
• Define comprehensive default values.
• Store parameters in multiple file formats (.toml, .json, .yaml, .ini and .py).
• Sensitive secrets like tokens and passwords can be stored in safe places like .secrets file or vault server.
• Parameters can optionally be stored in external services like Redis server.
• Simple feature flag system.
• Layered [environment] system.
• Environment variables can be used to override parameters.
• Support for .env files to automate the export of environment variables.
• Correct data types (even for environment variables).
• Have only one canonical settings module to rule all your instances.
• Drop in extension for Flaskapp.config object.
• Drop in extension for Djangoconf.settings object.
• Powerful $ dynaconf CLI to help you manage your settings via console.
• Customizable Validation System to ensure correct config parameters.
• Allow the change of dynamic parameters on the fly without the need to redeploy your application.
• Easily extensible
• 100% test coverage
• 100% documented

Read the Full Documentation at: http://dynaconf.readthedocs.io/

Repository: http://github.com/rochacbruno/dynaconf/

Getting Started

Installation

Python 3.x is required

$ pip install dynaconf

Default installation supports .toml, .py and .json file formats and also environment variables (.env supported)

Usage

Accessing config variables in your Python application

In your Python program wherever you need to access a settings variable you use the canonical object from dynaconf import settings:

NOTE: Read the full documentation for more examples like using Dynaconf with Flask or Django

Example of program to connect to some database

from some.db import Client

from dynaconf import settings               # The only object you need to import

conn = Client(
    username=settings.USERNAME,             # attribute style access
    password=settings.get('PASSWORD'),      # dict get style access
    port=settings['PORT'],                  # dict item style access
    timeout=settings.as_int('TIMEOUT'),     # Forcing casting if needed
    host=settings.get('HOST', 'localhost')  # Providing defaults
)

Where the settings values are stored

Dynaconf aims to have a flexible and usable configuration system. Your applications can be configured via a configuration files, through environment variables, or both. Configurations are separated into environments: [development], [staging], [testing] and [production]. The working environment is selected via an environment variable.

Sensitive data like tokens, secret keys and password can be stored in .secrets.* files and/or external storages like Redis or vault secrets server.

Besides the built-in optional support to redis as settings storage dynaconf allows you to create custom loaders and store the data wherever you want e.g: databases, memory storages, other file formats, nosql databases etc.

Working environments

At any point in time, your application is operating in a given configuration environment. By default there are four such environments:

• [development]
• [staging]
• [testing]
• [production]

You can also define [custom environment] and use the pseudo-envs [default] to provide comprehensive default values and [global] to provide global values to overrride in any other environment.

Without any action, your applications by default run in the [development] environment. The environment can be changed via the ÈNV_FOR_DYNACONF environment variable. For example, to launch an application in the [staging] environment, we can run:

export ENV_FOR_DYNACONF=staging

or

ENV_FOR_DYNACONF=staging python yourapp.py

NOTE: When using FLask Extension the environment can be changed via FLASK_ENV variable and for Django Extension you can use DJANGO_ENV.

The settings files

NOTE:Read the full documentaion about dynaconf CLI to learn how to automatically create the settings files for your project.

An optional settings.{toml|py|json|ini|yaml} file can be used to specify the configuration parameters for each environment. If it is not present, only the values from environment variables are used (.env file is also supported). Dynaconf searches for the file starting at the current working directory. If it is not found there, Dynaconf checks the parent directory. Dynaconf continues checking parent directories until the root is reached.

The recommended file format is TOML but you can choose to use any of .{toml|py|json|ini|yaml}.

The file must be a series of sections, at most one for [default], optionally one for each [environment], and an optional [global] section. Each section contains key-value pairs corresponding to configuration parameters for that [environment]. If a configuration parameter is missing, the value from [default] is used. The following is a complete settings.toml file, where every standard configuration parameter is specified within the [default] section:

NOTE: if the file format choosen is .py as it does not support sections you can create multiple files like settings.py for [default], development_settings.py, production_settings.py and global_settings.py. ATTENTION using .py is not recommended for configuration use TOML!

[default]
username = "admin"
port = 5000
host = "localhost"
message = "default message"
value = "default value"

[development]
username = "devuser"

[staging]
host = "staging.server.com"

[testing]
host = "testing.server.com"

[production]
host = "server.com"

[awesomeenv]
value = "this value is set for custom [awesomeenv]"

[global]
message = "This value overrides message of default and other envs"

The [global] pseudo-environment can be used to set and/or override configuration parameters globally. A parameter defined in a [global] section sets, or overrides if already present, that parameter in every environment. For example, given the following settings.toml file, the value of address will be "1.2.3.4" in every environment:

[global]
address = "1.2.3.4"

[development]
address = "localhost"

[production]
address = "0.0.0.0"

NOTE: The [env] name and first level variables are case insensitive as internally dynaconf will always use upper case, that means [development] and [DEVELOPMENT] are equivalent and address and ADDRESS are also equivalent. This rule does not apply for inner data structures as dictionaries and arrays.

Supported file formats

By default toml is the recommended format to store your configuration, however you can switch to a different supported format.

# If you wish to include support for more sources
pip3 install dynaconf[yaml|ini|redis|vault]

# for a complete installation
pip3 install dynaconf[all]

Once the support is installed no extra configuration is needed to load data from those files, dynaconf will search for settings files in the root directory of your application looking for the following files in the exact order below:

DYNACONF_LOADING_ORDER = [
 'settings.py',
 '.secrets.py',
 'settings.toml',
 '.secrets.toml',
 'settings.yaml',
 '.secrets.yaml',
 'settings.ini',
 '.secrets.ini',
 'settings.json',
 '.secrets.json',
 # redis server if REDIS_ENABLED_FOR_DYNACONF=true
 # vault server if VAULT_ENABLED_FOR_DYNACONF=true
 # other sources if custom loaders are defined
 # All environment variables prefixed with DYNACONF_
]

NOTE: Dynaconf works in an layered override mode based on the above order, so if you have multiple file formats with conflicting keys defined, the precedence will be based on the loading order.

Take a look at the example folder to see some examples of use with different file formats.

Sensitive secrets

Using .secrets files

To safely store sensitive data Dynaconf also searches for a .secrets.{toml|py|json|ini|yaml} file to look for data like tokens and passwords.

example .secrets.toml:

[default]
password = "sek@987342$"

The secrets file supports all the environment definitions supported in the settings file.

IMPORTANT: The reason to use a .secrets.* file is the ability to omit this file when commiting to the repository so a recommended .gitignore should include .secrets.* line.

Using Vault server

The vaultproject.io/ is a key:value store for secrets and Dynaconf can load variables from a Vault secret.

1. Run a vault server

Run a Vault server installed or via docker:

$ docker run -d -e 'VAULT_DEV_ROOT_TOKEN_ID=myroot' -p 8200:8200 vault

1. Install support for vault in dynaconf

$ pip install dynaconf[vault]

1. In your .env file or in exported environment variables define:

VAULT_ENABLED_FOR_DYNACONF=true
VAULT_URL_FOR_DYNACONF="http://localhost:8200"
VAULT_TOKEN_FOR_DYNACONF="myroot"

Now you can have keys like PASSWORD and TOKEN defined in the vault and dynaconf will read it.

To write a new secret you can use http://localhost:8200 web admin and write keys under the /secret/dynaconf secret database.

You can also use the Dynaconf writer via console

$ dynaconf write vault -s password=123456

Environment variables

overloading parameters via env vars

All configuration parameters, including custom environments and dynaconf configuration, can be overridden through environment variables.

To override the configuration parameter {param}, use an environment variable named DYNACONF_{PARAM}. For instance, to override the "HOST" configuration parameter, you can run your application with:

DYNACONF_HOST='otherhost.com' python yourapp.py

.env files

If you don't want to declare the variables on every program call you can run export DYNACONF_{PARAM} in your shell or put the values in a .env file located in the same directory as your settings files (the root directory of your application), variables in .env does not overrride existing environment variables.

IMPORTANT: Dynaconf will search for a .env located in the root directory of your application, if not found it will continue searching in parent directories until it reaches the root. To avoid conflicts we recommend to have a .env even if it is empty.

Precedence and type casting

Environment variables take precedence over all other configuration sources: if the variable is set, it will be used as the value for the parameter even if parameter exists in settings files or in .env.

Variable values are parsed as if they were TOML syntax. As illustration, consider the following examples:

# Numbers
DYNACONF_INTEGER=42
DYNACONF_FLOAT=3.14

# Text
DYNACONF_STRING=Hello
DYNACONF_STRING="Hello"

# Booleans
DYNACONF_BOOL=true
DYNACONF_BOOL=false

# Use extra quotes to force a string from other type
DYNACONF_STRING="'42'"
DYNACONF_STRING="'true'"

# Arrays must be homogenous in toml syntax
DYNACONF_ARRAY=[1, 2, 3]
DYNACONF_ARRAY=[1.1, 2.2, 3.3]
DYNACONF_ARRAY=['a', 'b', 'c']

# Dictionaries
DYNACONF_DICT={key="abc",val=123}

# toml syntax does not allow `None/null` values so use @none
DYNACONF_NONE='@none None'

# toml syntax does not allow mixed type arrays so use @json
DYNACONF_ARRAY='@json [42, 3.14, "hello", true, ["otherarray"], {"foo": "bar"}]'

NOTE: Older versions of Dynaconf used the @casting prefixes for env vars like export DYNACONF_INTEGER='@int 123' still works but this casting is deprecated in favor of using TOML syntax described above. To disable the @casting do export AUTO_CAST_FOR_DYNACONF=false

The global prefix

The DYNACONF_{param} prefix is set by GLOBAL_ENV_FOR_DYNACONF and serves only to be used in environment variables to override config values.

This prefix itself can be changed to something more significant for your application, however we recommend kepping DYNACONF_{param} as your global env prefix.

NOTE: See the Configuring dynaconf section in documentation to learn more on how to use .env variables to configure dynaconf behavior.

Flask Extension

Dynaconf provides a drop in replacement for app.config.

As Flask encourages the composition by overriding the config_class attribute this extension follows the patterns of Flask and turns your Flask's app.config in to a dynaconf instance.

Initialize the extension

Initialize the FlaskDynaconf extension in your app

from flask import Flask
from dynaconf import FlaskDynaconf

app = Flask(__name__)
FlaskDynaconf(app)

You can optionally use init_app as well.

Use FLASK_ environment variables

Then the app.config will work as a dynaconf.settings instance and FLASK_ will be the global prefix for exporting environment variables.

Example:

export FLASK_DEBUG=true              # app.config.DEBUG
export FLASK_INTVALUE=1              # app.config['INTVALUE']
export FLASK_MAIL_SERVER='host.com'  # app.config.get('MAIL_SERVER')

Settings files

You can also have settings files for your Flask app, in the root directory (the same where you execute flask run) put your settings.toml and .secrets.toml files and then define your environments [default], [development] and [production].

To switch the working environment the FLASK_ENV variable can be used, so FLASK_ENV=development to work in development mode or FLASK_ENV=production to switch to production.

IMPORTANT: To use $ dynaconf CLI the FLASK_APP must be defined.

IF you don't want to manually create your config files take a look at the CLI

Django Extension

Dynaconf a drop in replacement to django.conf.settings.

Following this pattern recommended pattern this extension makes your Django's conf.settings in to a dynaconf instance.

Initialize the extension

In your django project's settings.py include:

INSTALLED_APPS = [
    'dynaconf.contrib.django_dynaconf',
    ...
]

NOTE: The extension must be included as the first INSTALLED_APP of the list

Use DJANGO_ environment variables

Then django.conf.settings will work as a dynaconf.settings instance and DJANGO_ will be the global prefix to export environment variables.

Example:

export DJANGO_DEBUG=true     # django.conf.settings.DEBUG
export DJANGO_INTVALUE=1     # django.conf.settings['INTVALUE]
export DJANGO_HELLO="Hello"  # django.conf.settings.get('HELLO)

Settings files

You can also have settings files for your Django app, in the root directory (the same where manage.py is located) put your settings.toml and .secrets.toml files and then define your environments [default], [development] and [production].

To switch the working environment the DJANGO_ENV variable can be used, so DJANGO_ENV=development to work in development mode or DJANGO_ENV=production to switch to production.

IMPORTANT: To use $ dynaconf CLI the DJANGO_SETTINGS_MODULE must be defined.

IF you don't want to manually create your config files take a look at the CLI

NOTE: It is recommended that all the django's internal config vars should be kept in the settings.py of your project, then application specific values you can place in dynaconf's settings.toml in the root (same folder as manage.py). You can override settings.py values in the dynaconf settings file as well.

How to contribute

In github repository issues and Pull Request are welcomed!

• New implementations
• Bug Fixes
• Bug reports
• More examples of use in /example folder
• Documentation
• Feedback as issues and comments ot joining dynaconf on Telegram or #dynaconf on freenode
• Donation to rochacbruno [at] gmail.com in PayPal

Read the docs

Documentation: http://dynaconf.readthedocs.io/

Repository: http://github.com/rochacbruno/dynaconf/