Week 9: The Cloud

image: mnsc http://www.flickr.com/photos/mnsc/2768391365/ - CC-BY

You've built your app, tested it, now it's time to go live

What are your options?

It Depends

In the old days™ you had one option

Buy a server, build it and host it yourself

You have total control

• buy exactly the hardware you want
• run only the services you need

You also have total responsibility

• when something breaks, you have to fix it
• you bear all the costs yourself

Expensive

• Server-class hardware $2k-$10k or more
• Systems Administrator $90K/year

Inefficient

• Most web sites don't get enough traffic to tax a good server
• Web traffic tends to be assymetrical
• Systems administration tasks also highly assymetrical

A problem of resource utilization

VPS (Virtual Private Server)

A part of a server (or perhaps an entire server) purchased from a provider.

You pay for only a portion of a server and a portion of a systems administrator.

You retain control of the system

You also retain responsibility for everything above the bare iron.

Benefits

• Reduced cost ($30-$100+/month vs server and salary)
• Reduced burden (The provider handles hardware upkeep and low-level maintenance)
• Retain control of your full software stack

Drawbacks

• You install and maintain the web stack (requires knowledge)
• You lose control over resource utilization
• Your resources are still fixed (always the same size)

Shared Hosting

You pay a provider to set you up with a [django/flask/pyramid/etc.] system.

Hardware and most software maintenance is provided

You are able to install some (but perhaps not all) add-ons

This solution is very popular in the PHP world

Much less so with Python. Why?...

Benefits

• Enormously less expensive ($5-$15+/month)
• Much lower maintenance burden
• Simplified installation process

Drawbacks

• Tight resource restrictions (cpu, ram, disk space)
• Little to no control over most of the stack
• Reduced availability of some frameworks or packages

And still, no ability to grow if you need

The Cloud

(cue fanfare)

You don't know today what you will need tomorrow

Today your website is getting 100-500 unique visitors

Tomorrow you might have 10,000, 100,000. Who knows?

Should you have to buy enough hardware to handle that traffic today?

Cloud computing offers rapid deployment solutions so you can scale at will

Really, it differs from place to place.

Some are more do-it-yourself (Amazon EC2, Rackspace Cloud)

Some are more automated (Heroku, Elastic Beanstalk, AppEngine)

All try to abstract common deployment tasks to make it easy to repeat

So, how does that work?

Fabric is not a cloud service. Instead, it's a tool built to help developers simplify the process of deploying complex apps to a server.

It can be used in any setup where you have ssh access to the filesystem of the remote server.

Your classmate Austin used it a couple of weeks back to deploy Django to his bluebox VM.

Today, he's going to share that experience with you...

I tried a number of cloud providers

This was hands-down the easiest.

You'll need a Heroku account to do anything, so the first step is to get that

Go to http://www.heroku.com

Click on 'Sign Up' and enter your email address

When the email arrives, click the link and create your password

Once you've signed up, you'll see your 'dashboard' page with tips on getting started.

Like pretty much all the 'cloud' providers out there, Heroku has some command-line tools you need to use.

You can find them at https://toolbelt.heroku.com/

Download and install the package, and then login:

$ heroku login
Email: your-email@your.domain.com
Password: <fill it in>

The tool will find, or help you create, an ssh public key

As an exercise, I decided to deploy the djangor micro-blog app we created in class.

The first step was to clone the app, then create a local branch for deployment:

$ mkdir heroku-test
$ cd heroku-test
$ git clone git@github.com:cewing/training.django_microblog.git
...
$ cd training.django_microblog
$ git checkout -b heroku-deploy
Switched to a new branch 'heroku-deploy'
$

Again, like many cloud providers Heroku uses virtualenvs to ensure it's installed correctly

I set up a python 2.7 virtualenv right in my git repository:

$ ~/pythons/bin/virtualenv-2.7 --distribute venv
...
$ source venv/bin/activate
(venv)$

I don't want to check that virtualenv into git, so I add venv to my .gitignore file.

That way, git will ignore that directory and everything in it.

For Heroku to work, it needs to know what packages you'll need installed.

We can use pip to take care of this:

(venv)$ pip install Django=1.4.5 psycopg2 dj-database-url

Psycopg2 is a DBAPI connector for PostgreSQL. Heroku requires Postgresql

dj-database-url allows the Django DB settings to come from an env variable.

Heroku uses pip too. It uses a file called requirements.txt to know what to do.

You create that file:

(venv)$ pip freeze > requirements.txt

Then, add the file to your repository and commit:

(venv)$ git add requirements.txt
(venv)$ git commit -m "setting requirements for heroku"

To adapt Django to the Heroku environment, we need to add the following to the end of our settings.py file:

# Parse database configuration from $DATABASE_URL
import dj_database_url
DATABASES['default'] =  dj_database_url.config()

# Honor the 'X-Forwarded-Proto' header for request.is_secure()
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')

Commit these changes to your heroku-deploy branch.

Local development settings are different. You can use different settings for production and development

Finally, we need to create a file named Procfile.

Heroku uses this to learn about the processes we want running.

Lines in the file take the form process_type: command,

Create the file Procfile and add the following text:

web: python manage.py runserver 0.0.0.0:$PORT --noreload

Then, add and commit that file to the repository.

At this point, we're ready to go.

First, we create a new app in heroku with our repo:

(venv)$ heroku create
Creating fierce-plains-6505... done, stack is cedar
http://fierce-plains-6505.herokuapp.com/ | git@heroku.com:fierce-plains-6505.git
Git remote heroku added
(venv)$

Then, deploy it by 'pushing' to the heroku remote (master branch):

(venv)$ git push heroku heroku-deploy:master

Heroku works like github, in a way.

When our repository is pushed, a hook script detects the update and starts working.

• Heroku detects that we are building a python project
• A python virtualenv is created
• pip installs the dependencies in requirements.txt
• Heroku further detects that we are building a Django app and runs collectstatic
• Our Procfile is read, and data about the processes we want is written to the environment

Heroku will not run syncdb for us. We have to do that on our own.

Heroku does provide us with a way to run one-off commands on our server, though:

(venv)$ heroku run python manage.py syncdb

This command is run through an ssh tunnel. We can interact with it.

We can use other commands, like shell with heroku run.

All we have to do now is start a process so we can see our work:

(venv)$ heroku ps:scale web=1
Scaling web processes... done, now running 1
(venv)$ heroku ps
=== web: `python manage.py runserver 0.0.0.0:$PORT --noreload`
web.1: up 2013/03/05 06:28:13 (~ 21m ago)
(venv)$ heroku open

That last bit will automatically open a web browser pointing at the URL where may be seen.

Heroku does not want you to point A record DNS names at it's services.

Using www.mydomain.com is okay, but mydomain.com is not.

They also don't want you to use IP addresses, since their architecture means IP addresses change.

I set up a CNAME record for microblog.crisewing.com. It points to the URL opened when I type heroku open.

So long as I keep this heroku app, that domain name will not change.

This is but one example of a cloud deployment.

It is considerably easier to do than most other cloud deployments.

It is also considerably more constrained than other deployments.

When you are making choices about deployment, you must take into consideration your needs, both now and in the future:

• What type of Framework will you use?
• What type of Database will you use?
• What growth do you expect to experience (best and worst case)?
• How much control do you want over all the processes that make your website run?
• How much time/expertise do you have (or can you afford to acquire)?

Carefully consider these questions, and you will find an appropriate solution.

For the rest of today, we work on your projects.