Install Pyramid on Ubuntu 12.04 LTS in the Rackspace Cloud

Check the Installed Python Version

python --version

You should see the following output:

Python 2.7.3

Install Prerequisites

apt-get install python-setuptools python-pip python-virtualenv virtualenvwrapper

Install Prerequisites for Pyramid Speedups

apt-get install gcc cpp libc6-dev python2.7-dev

Install nginx

apt-get install nginx nginx-full nginx-common

Create a wwwuser that waitress (the web server) will run as

useradd wwwuser -d /home/wwwuser -k /etc/skel -m -s /bin/bash -U

Setup the Virtual Environment

mkdir -p /var/www/delixus.com
mkdir /var/www/environments
cd /var
chown -R wwwuser:wwwuser www

We are now going to change users to wwwuser user.

su - wwwuser
cd /var/www/environments
virtualenv env_delixus

Install Pyramid

You must perform the following steps as the wwwuser user.

cd /var/www/environments/env_delixus
source bin/activate

You should see the environment name as the prefix in the command prompt, such as:

(env_delixus)wwwuser@ws2:

easy_install Pyramid
pip install waitress

Checkout the Pyramid Project

cd /var/www/delixus.com

Change the SVN checkout command to something that matches your server. If you use git, then change appropriately.

svn checkout https://repo.company.com/source/delixus/tags/1.0 .

Install the delixus.com Pyramid project

cd /var/www/delixus.com/delixus
vi production.ini

Under [app:main], add a [server:main] configuration as follows:


# http://docs.pylonsproject.org/projects/waitress/en/latest/arguments.html
[server:main]
use = egg:waitress#main
host = 127.0.0.1
port = %(http_port)s
# default # of threads = 4
threads = 8
url_scheme = http

I don’t think you need to install the development version of the site, but it seems to be the only way that I get everything to work while debugging…go figure.

python setup.py develop
pserve development.ini

Then open the site in a text-based web browser.

links http://localhost:6543

You should be able to view your site at this point.

Now, let’s install the production version of the site.

python setup.py install

Start Waitress

First we’re going to start and test waitress, then we’ll start it as a deamon.

pserve production.ini http_port=5000
links http://localhost:5000

Again, you should be able to view your site.

pserve production.ini start --daemon --pid-file=/var/www/5000.pid \
--log-file=/var/www/5000.log --monitor-restart http_port=5000
pserve production.ini start --daemon --pid-file=/var/www/5001.pid \
--log-file=/var/www/5001.log --monitor-restart http_port=5001

Check the waitress process.

ps -ef | grep pserve

You should see the pserve process running.

Configure nginx as a Proxy for Waitress

The following steps must be performed as root.

cd /etc/nginx/sites-available
vi delixus

Paste the following into the delixus.conf file.


upstream delixus-site {
    server 127.0.0.1:5000;
    server 127.0.0.1:5001;
}

server {
    listen 80;
    server_name  localhost www.delixus.com delixus.com;

    access_log  /var/log/nginx/delixus.com-access.log;

    location / {
        proxy_set_header        Host $host;
        proxy_set_header        X-Real-IP $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header        X-Forwarded-Proto $scheme;

        client_max_body_size    10m;
        client_body_buffer_size 128k;
        proxy_connect_timeout   60s;
        proxy_send_timeout      90s;
        proxy_read_timeout      90s;
        proxy_buffering         off;
        proxy_temp_file_write_size 64k;
        proxy_pass http://delixus-site;
        proxy_redirect          off;
    }

    location /static {
        root            /var/www/delixus.com/delixus/delixus;
        expires         30d;
        add_header      Cache-Control public;
        access_log      off;
    }
}

rm /etc/nginx/sites-enabled/default
ln -s /etc/nginx/sites-available/delixus /etc/nginx/sites-enabled/delixus
service nginx stop
service nginx start

A good next step at this point is to setup Supervisor to control pserve/waitress.

Important update to earlier post on MySQL Installation

I have updated an earlier post on how to install MySQL 5.5 on Ubuntu 10.04 LTS.

Importantly, the new instructions include how to add the MySQL 5.5 libs to the loader path. This is very important as it’s highly likely that you’ll build software that depends on these MySQL libraries.

The updated instructions can be viewed at: MySQL 5.5 on Ubuntu 10.04 LTS.

Output XML using Pyramid and Mako

I recently had to connect a .NET desktop application to a web service that was built using Pyramid. My first thought was to use JSONP. However, JSONP is only useful for small amounts of data…and I had a lot of data to send/receive. Obviously, communication via an XmlHttpRequest with JSON was not an option due to limitations imposed by the Same Origin Policy (ref: http://en.wikipedia.org/wiki/Same_origin_policy).

In the end, we chose to use our existing Mako template infrastructure to render XML that the .NET clients could consume. If you’re a .NET developer, then you know that XML parsing in .NET is superb, which is why we went with XML rather than JSON over HTTP.

On the Pyramid side, there were really only 2 steps:

  1. Create Mako templates that would render XML
  2. Set the content type to text/xml

Render XML with Mako

This step was easy, we just created templates that use XML instead of XML.

<?xml version=”1.0″ encoding=”utf-8″?>
<companies>
% if companies:
% for row in companies:
<company>
<id>${row[‘id’]}</id>
<name>${row[‘name’]}</name>
</company>
% endfor
% else:
${result}
% endif
</companies>

Set the Response Content Type

This was also a very easy step. We just set the content_type immediately before the call to return. Importantly, the code below causes Pyramid to return the XML template with a content type of text/xml rather than the default text/html.

request.response.content_type = “text/xml”
return {‘companies’: companies}

Summary

Rendering XML with Pyramid and Mako was exceptionally simple. You can reuse the same view and url, and simply change the the template based on how the view was called.

One last note. You’ll notice that one negative side effect we have not corrected for is the use of \n\r in the rendered XML. This of course bloats network traffic, uses more memory on the client-side (if you’re using a DOM parser), and is all around crappy for all but the smallest of files. Next step will be to create some type of processor to eliminate the newline character.