For part 1

Refactoring settings

Environment variable-based settings are a big thing in the 12-factor app. I have found this incredibly confusing because you have to store the configurations somewhere else besides the ephemeral environment. Those environment variables get populated from somewhere, and I haven’t seen a good example of how to manage these across the development lifecycle.

So I’m making it up and documenting it here.

Also, another goal is deploy these changes before we fully convert to Docker. There will be some changes that are necessary for now, but will no longer necessary in the future.

If you don’t like it, why do it?

It’s not that I don’t like environment variable-based settings. It is that they never really made sense to me in the development lifecycle we use. However, the introduction of Docker changes things a bit.

In our existing setup, there isn't a clear cut place to set all the environment variables and make it easy to manage them. With Docker, it is clear that they should go in the Dockerfile.

So now we have three ways to set up the configuration of an environment:

1. Environment variables for basic settings
2. A specific settings file for complex settings
3. A .env file that we could programmatically add in a deployment script

This gives us lots of flexibility to configure local development, user testing, automated testing (CI/CD), and production environments.

The tool: django-environ

We added Django environ to our dependency list. It is pretty straightforward to use.

Here’s what we did:

Add .env to .gitignore

Django environ allows you to store environmental variables in a .env file. We don’t want this stored in our repo, so we need to tell git to ignore it.

Add to requirements

We updated our requirements.txt file with django-environ==0.4.1.

Modify our settings structure

Like many Django projects, our settings is a directory:

settings/
 ├ __init__.py
 ├ base.py
 ├ local_settings.template.py
 ├ production.py
 ├ test.template.py
 └ vagrant.py

A couple of notes:

__init__.py This attempts to import local_settings.py if it exists. Otherwise imports base.py.

base.py All of our settings are in here.

local_settings.template.py A new developer will copy this file to local_settings.py for local development. It allows for the manipulation of the core settings in ways that are advantageous for development. local_settings.py is ignored by git.

We don’t plan on stopping using a local settings file, although we may use it less.

production.py This file contains production-specific settings.

We won’t need this file any more.

test.template.py This template file is used to create test.py when we generate a test instance.

We may not need this any more. When we decide on if we will change anything on how our test server is set up, we will know.

vagrant.py This is for developers using vagrant for development. Not all the developers use vagrant, and we don’t force them to.

This will probably still be used.

Modify settings

At the top of settings/base.py we have:

import environ

PROJECT_ROOT = environ.Path(__file__) - 2  # two folders back (/project/settings/base.py - 2 = /)
env = environ.Env()

if os.path.exists(PROJECT_ROOT('.env')):
    env.read_env(PROJECT_ROOT('.env'))

This snippet sets up the PROJECT_ROOT variable used to modify paths throughout the file, and then checks for a .env file and reads it.

We do this check because if you call read_env() and the file doesn’t exist, it raises a UserWarning. We don't currently plan on using a .env in production, so this warning will appear every time gunicorn starts or restarts. This is just noise, so we do a manual check for the .env file first.

Here are the settings we changed initially:

DEBUG = env('DEBUG', default=False)

DATABASES = {
    'default': env.db_url(default="postgresql://postgres:postgres@localhost:5432/education")
}

CACHES = {
    'default': env.cache_url(default='dummycache://')
}

# This may change when we decide on our media handling
SFTP_STORAGE_HOST = env.dict('SFTP_STORAGE_HOST', default='prod-cache-01')
SFTP_STORAGE_PARAMS = env.dict('SFTP_STORAGE_PARAMS', default={'username': 'natgeo'})

# Google Analytics information
GAQ_ACCOUNT = env('GAQ_ACCOUNT', default='UA-xxxxxxx-x’)
GAQ_DOMAIN = env('GAQ_DOMAIN', default=SITE_URL)

# Elasticsearch information
ES_HOST = [env('ES_HOST', default='localhost')]
ES_PORT = env('ES_PORT', default='9200')
ES_INDEX = env('ES_INDEX', default='ngs')

But what about the SECRET_KEY? In Django, the SECRET_KEY is an important security value. So why isn't it included in the environment variables? Because everyone needs it and it never changes. Any environment without that key will not be able to log users in (using a recent backup of the production database).

So if there are better ways to more securely handle the use and distribution of the SECRET_KEY, I am all ears. Or mostly ears and a lot of forehead.

Environment variables and types

Here is a couple of things we had to figure out, or at least weren't very clear in the documentation:

Want some extra options in your cache settings?

After the URL, include the extra OPTIONS as query parameters:

CACHE_URL=rediscache://127.0.0.1:6379/0?CLIENT_CLASS=site_ext.cacheclient.GracefulClient

Now in your settings, env.cache() returns:

{'BACKEND': 'django_redis.cache.RedisCache',
 'LOCATION': 'redis://127.0.0.1:6379/0',
 'OPTIONS': {'CLIENT_CLASS': 'site_ext.cacheclient.GracefulClient'}}

Want to pass in a dict?

The keys and values are comma-separated key=value strings, like key1=value1,key2=value2. Then assign that string to the environment variable like so:

DICT_VAR=key1=value1,key2=value2