Skip to content

Latest commit

 

History

History
477 lines (332 loc) · 13.8 KB

File metadata and controls

477 lines (332 loc) · 13.8 KB

Flask

The package dockerflow.flask package implements various tools to support Flask based projects that want to follow the Dockerflow specs:

  • A Python logging formatter following the mozlog format.
  • A Flask extension implements:
    • Emitting of request.summary log records based on request specific data.
    • Views for health monitoring:
      • /__version__ - Serves a version.json file
      • /__heartbeat__ - Runs the configured Dockerflow checks
      • /__lbheartbeat__ - Retuns a HTTP 200 response
    • Signals for passed and failed heartbeats.
    • Built-in Dockerflow checks for SQLAlchemy and Redis connections and validating Alembic migrations.
    • Hooks to add custom Dockerflow checks.
  • Adds request_id to the flask.g application namespace when it isn't already set
.. seealso::

    For more information see the :doc:`API documentation <api/flask>` for
    the ``dockerflow.flask`` module.

Setup

To install python-dockerflow's Flask support please follow these steps:

  1. In your code where your Flask application lives set up the dockerflow Flask extension:

    from flask import Flask
    from dockerflow.flask import Dockerflow
    
    app = Flask(__name__)
    dockerflow = Dockerflow(app)
    
  2. Make sure the app root path is set correctly as this will be used to locate the version.json file that is generated by CircleCI or another process during deployment.

    .. seealso:: :ref:`flask-versions` for more information
    
    
  3. Configure logging to use the JsonLogFormatter logging formatter for the request.summary logger (you may have to extend your existing logging configuration), see :ref:`flask-logging` for more information.

Configuration

Accept its configuration through environment variables.

There are several options to handle configuration values through environment variables when configuring Flask.

os.environ

The simplest is to use Python's os.environ object to access environment variables for settings and other variables, e.g.:

MY_SETTING = os.environ.get('FLASK_MY_SETTING', 'default value')

The downside of that is that it nicely works only for string based variables, since that's what os.environ returns.

python-decouple

A good replacement is python-decouple as it's agnostic to the framework in use and offers casting the returned value to the type wanted, e.g.:

from decouple import config

MY_SETTING = config('FLASK_MY_SETTING', default='default value')
DEBUG = config('FLASK_DEBUG', default=False, cast=bool)

As you can see the DEBUG configuration value would be populated from the FLASK_DEBUG environment variable but also be cast as a boolean (while considering the string values '1', 'yes', 'true' and 'on' as truthy values, and similar for falsey values).

flask-environ

flask-environ follows similar patterns as python-decouple but implements specific casters for typical Flask configuration values. E.g.:

from flask import Flask
from flask_environ import get, collect, word_for_true

app = Flask(__name__)

app.config.update(collect(
    get('DEBUG', default=False, convert=word_for_true),
    get('HOST', default='127.0.0.1'),
    get('PORT', default=5000, convert=int),
    get('SECRET_KEY',
        'SQLALCHEMY_DATABASE_URI',
        'TWITTER_CONSUMER_KEY',
        'TWITTER_CONSUMER_SECRET',
    ),
))

Flask-Env

If you need to solve more complex configuration scenarios there are tools like Flask-Env which allows loading settings for different environments (e.g. dev, stage, prod) via environment variables. It provides a small Python meta class to allow setting up the configuration values:

E.g. in a config.py file next to your application:

from flask_env import MetaFlaskEnv

class Dev(metaclass=MetaFlaskEnv):
    DEBUG = True
    PORT = 5000

class Prod(Dev):
    DEBUG = False

Then in your application code:

import os
from flask import Flask

app = Flask(__name__)
app.config.from_object(os.environ.get('FLASK_CONFIG', 'config.Dev'))

In that example the configuration class that is given in the FLASK_CONFIG environment variable would be used to update the default Flask configuration values while allowing to override the values via environment variables.

It's recommended to use the Flask-Env feature to define a prefix for the environment variable it uses to check, e.g.:

from flask_env import MetaFlaskEnv

class Dev(metaclass=MetaFlaskEnv):
    ENV_PREFIX = 'ACME_'
    DEBUG = True

To override the config value of DEBUG the environment variable would be called ACME_DEBUG.

PORT

Listen on environment variable $PORT for HTTP requests.

Depending on which WSGI server you are using to run your Python application there are different ways to accept the :envvar:`PORT` as the port to launch your application with.

It's recommended to use port 8000 by default.

Gunicorn

Gunicorn automatically will bind to the hostname:port combination of 0.0.0.0:$PORT if it find the :envvar:`PORT` environment variable. That means running gunicorn is as simple as using this, for example:

gunicorn myproject:app --workers 4
.. seealso::

    The `full gunicorn documentation <http://docs.gunicorn.org/>`_
    for more details.

uWSGI

For uWSGI all you have to do is to bind on the :envvar:`PORT` when you define the uwsgi.ini, e.g.:

[uwsgi]
http-socket = :$(PORT)
master = true
processes = 4
module = myproject:app
chdir = /app
enable-threads = True
.. seealso::

    The `full uWSGI documentation <http://uwsgi-docs.readthedocs.io/>`_
    for more details.

Versions

Must have a JSON version object at /app/version.json.

Dockerflow requires writing a version object to the file /app/version.json as seen from the docker container to be served under the URL path /__version__.

To facilitate this python-dockerflow comes with a Flask view to read the file under path the parent directory of the Flask app root. See the :class:`Flask API docs <~flask.Flask>` for more information about the app root path.

If you'd like to override the location from which the view is reading the version.json file from, simply override the optional version_path parameter to the :class:`~dockerflow.flask.app.Dockerflow` class, e.g.:

from flask import Flask
from dockerflow.flask import Dockerflow

app = Flask(__name__)
dockerflow = Dockerflow(app, version_path='/app')

Alternatively if you'd like to completely override the way the version information is read use the :meth:`~dockerflow.flask.app.Dockerflow.version_callback` decorator to decorate a callback that gets the version_path value passed. E.g.:

import json
from flask import Flask
from dockerflow.flask import Dockerflow

app = Flask(__name__)
dockerflow = Dockerflow(app)

@dockerflow.version_callback
def my_version(root):
    return json.loads(os.path.join(root, 'acme_version.json'))

Health monitoring

Health monitoring happens via three different views following the Dockerflow spec:

.. http:get:: /__version__

   The view that serves the :ref:`version information <flask-versions>`.

   **Example request**:

   .. sourcecode:: http

      GET /__version__ HTTP/1.1
      Host: example.com

   **Example response**:

   .. sourcecode:: http

      HTTP/1.1 200 OK
      Vary: Accept-Encoding
      Content-Type: application/json

      {
        "commit": "52ce614fbf99540a1bf6228e36be6cef63b4d73b",
        "version": "2017.11.0",
        "source": "https://github.com/mozilla/telemetry-analysis-service",
        "build": "https://circleci.com/gh/mozilla/telemetry-analysis-service/2223"
      }

   :statuscode 200: no error
   :statuscode 404: a version.json wasn't found

.. http:get:: /__heartbeat__

   The heartbeat view will go through the list of registered Dockerflow
   checks, run each check and add their results to a JSON response.

   The view will return HTTP responses with either an status code of 200 if
   all checks ran successfully or 500 if there was one or more warnings or
   errors returned by the checks.

   **Built-in Dockerflow checks:**

   There are a few built-in checks that are automatically added to the list
   of checks if the appropriate Flask extension objects are passed to
   the :class:`~dockerflow.flask.app.Dockerflow` class during instantiation.

   For detailed examples please see the API documentation for the built-in
   :ref:`Flask Dockerflow checks <flask-checks>`.

   **Custom Dockerflow checks:**

   To write your own custom Dockerflow checks simply write a function
   that returns a list of one or many check message instances representing
   the severity of the check result. The :mod:`dockerflow.flask.checks`
   module contains a series of predefined check messages for the
   severity levels: :class:`~dockerflow.flask.checks.Debug`,
   :class:`~dockerflow.flask.checks.Info`,
   :class:`~dockerflow.flask.checks.Warning`,
   :class:`~dockerflow.flask.checks.Error`,
   :class:`~dockerflow.flask.checks.Critical`.

   Here's an example of a check that handles various levels of exceptions
   from an external storage system with different check message::

       from dockerflow.flask import checks, Dockerflow

       app = Flask(__name__)
       dockerflow = Dockerflow(app)

       @dockerflow.check
       def storage_reachable():
           result = []
           try:
               acme.storage.ping()
           except SlowConnectionException as exc:
               result.append(checks.Warning(exc.msg, id='acme.health.0002'))
           except StorageException as exc:
               result.append(checks.Error(exc.msg, id='acme.health.0001'))
           return result

   Notice the use of the :meth:`~dockerflow.flask.app.Dockerflow.check`
   decorator to mark the check to be used.

   **Example request**:

   .. sourcecode:: http

      GET /__heartbeat__ HTTP/1.1
      Host: example.com

   **Example response**:

   .. sourcecode:: http

      HTTP/1.1 500 Internal Server Error
      Vary: Accept-Encoding
      Content-Type: application/json

      {
        "status": "warning",
        "checks": {
          "check_debug": "ok",
          "check_sts_preload": "warning"
        },
        "details": {
          "check_sts_preload": {
            "status": "warning",
            "level": 30,
            "messages": {
              "security.W021": "You have not set the SECURE_HSTS_PRELOAD setting to True. Without this, your site cannot be submitted to the browser preload list."
            }
          }
        }
      }

   :statuscode 200: no error
   :statuscode 500: there was a warning or error

.. http:get:: /__lbheartbeat__

   The view that simply returns a successful HTTP response so that a load
   balancer in front of the application can check that the web application
   has started up.

   **Example request**:

   .. sourcecode:: http

      GET /__lbheartbeat__ HTTP/1.1
      Host: example.com

   **Example response**:

   .. sourcecode:: http

      HTTP/1.1 200 OK
      Vary: Accept-Encoding
      Content-Type: application/json

   :statuscode 200: no error

Logging

Dockerflow provides a :class:`~dockerflow.logging.JsonLogFormatter` Python logging formatter class.

To use it, put something like this BEFORE your Flask app is initialized for at least the request.summary logger:

from logging.conf import dictConfig

dictConfig({
    'version': 1,
    'formatters': {
        'json': {
            '()': 'dockerflow.logging.JsonLogFormatter',
            'logger_name': 'myproject'
        }
    },
    'handlers': {
        'console': {
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
            'formatter': 'json'
        },
    },
    'loggers': {
        'request.summary': {
            'handlers': ['console'],
            'level': 'DEBUG',
        },
    }
})

Static content

To properly serve static content it's recommended to use Whitenoise. It contains a WSGI middleware that is able to serve the files that Flask usually serves under the static URL path (Flask app parameter static_url_path) from the Flask app's static folder (static_folder) but with far-future headers and proper response headers for the CDNs.

For more information see the documentation dedicated to using :doc:`Whitenoise with Flask <whitenoise:flask>`.

Another great adition (especially if no JavaScript based build system is used like webpack) is using Flask-Assets, a Flask extension based on the webassets management tool. Since it also uses the Flask app's static folder as the output directory by default both work well together.