Install Pyramid on Ubuntu 12.04 LTS in the Rackspace Cloud

Check the Installed Python Version

python --version

You should see the following output:

Python 2.7.3

Install Prerequisites

apt-get install python-setuptools python-pip python-virtualenv virtualenvwrapper

Install Prerequisites for Pyramid Speedups

apt-get install gcc cpp libc6-dev python2.7-dev

Install nginx

apt-get install nginx nginx-full nginx-common

Create a wwwuser that waitress (the web server) will run as

useradd wwwuser -d /home/wwwuser -k /etc/skel -m -s /bin/bash -U

Setup the Virtual Environment

mkdir -p /var/www/delixus.com
mkdir /var/www/environments
cd /var
chown -R wwwuser:wwwuser www

We are now going to change users to wwwuser user.

su - wwwuser
cd /var/www/environments
virtualenv env_delixus

Install Pyramid

You must perform the following steps as the wwwuser user.

cd /var/www/environments/env_delixus
source bin/activate

You should see the environment name as the prefix in the command prompt, such as:

(env_delixus)wwwuser@ws2:

easy_install Pyramid
pip install waitress

Checkout the Pyramid Project

cd /var/www/delixus.com

Change the SVN checkout command to something that matches your server. If you use git, then change appropriately.

svn checkout https://repo.company.com/source/delixus/tags/1.0 .

Install the delixus.com Pyramid project

cd /var/www/delixus.com/delixus
vi production.ini

Under [app:main], add a [server:main] configuration as follows:


# http://docs.pylonsproject.org/projects/waitress/en/latest/arguments.html
[server:main]
use = egg:waitress#main
host = 127.0.0.1
port = %(http_port)s
# default # of threads = 4
threads = 8
url_scheme = http

I don’t think you need to install the development version of the site, but it seems to be the only way that I get everything to work while debugging…go figure.

python setup.py develop
pserve development.ini

Then open the site in a text-based web browser.

links http://localhost:6543

You should be able to view your site at this point.

Now, let’s install the production version of the site.

python setup.py install

Start Waitress

First we’re going to start and test waitress, then we’ll start it as a deamon.

pserve production.ini http_port=5000
links http://localhost:5000

Again, you should be able to view your site.

pserve production.ini start --daemon --pid-file=/var/www/5000.pid \
--log-file=/var/www/5000.log --monitor-restart http_port=5000
pserve production.ini start --daemon --pid-file=/var/www/5001.pid \
--log-file=/var/www/5001.log --monitor-restart http_port=5001

Check the waitress process.

ps -ef | grep pserve

You should see the pserve process running.

Configure nginx as a Proxy for Waitress

The following steps must be performed as root.

cd /etc/nginx/sites-available
vi delixus

Paste the following into the delixus.conf file.


upstream delixus-site {
    server 127.0.0.1:5000;
    server 127.0.0.1:5001;
}

server {
    listen 80;
    server_name  localhost www.delixus.com delixus.com;

    access_log  /var/log/nginx/delixus.com-access.log;

    location / {
        proxy_set_header        Host $host;
        proxy_set_header        X-Real-IP $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header        X-Forwarded-Proto $scheme;

        client_max_body_size    10m;
        client_body_buffer_size 128k;
        proxy_connect_timeout   60s;
        proxy_send_timeout      90s;
        proxy_read_timeout      90s;
        proxy_buffering         off;
        proxy_temp_file_write_size 64k;
        proxy_pass http://delixus-site;
        proxy_redirect          off;
    }

    location /static {
        root            /var/www/delixus.com/delixus/delixus;
        expires         30d;
        add_header      Cache-Control public;
        access_log      off;
    }
}

rm /etc/nginx/sites-enabled/default
ln -s /etc/nginx/sites-available/delixus /etc/nginx/sites-enabled/delixus
service nginx stop
service nginx start

A good next step at this point is to setup Supervisor to control pserve/waitress.

Advertisements

Install Python 2.7.2 from source on Ubuntu 10.04 LTS

The first thing I did was to create a wwwuser that I plan to run pyramid under. As a result, I am intentionally installing Python 2.7.2 under only 1 user account, and am leaving the system wide python installation unchanged.

useradd wwwuser
passwd wwwuser
cd /home
mkdir wwwuser
chown wwwuser:wwwuser wwwuser

Copy all of the hidden files into the /home/wwwuser folder. I did this from my desktop files.

vi /etc/passwd
Update the shell to be:
/bin/bash

su - wwwuser
ln -s .profile .bash_profile
mkdir bin
mv Python-2.7.2.tgz bin
cd bin
tar -xzf Python-2.7.2.tgz
cd Python-2.7.2
./configure --prefix /home/wwwuser/bin/Python-2.7.2
make
make install

vi ~/.profile
Update path to:
PATH="$HOME/bin:/home/wwwuser/bin/Python-2.7.2/bin:$PATH"

source ~/.profile

which python
/home/wwwuser/bin/Python-2.7.2/bin/python

python --version

The output of the python --versioncommand should now be Python 2.7.2.

Install Tornado on Windows 7

Please see my prior post on how to Setup virtualenv on Windows.

Open PowerShell, then enter the following commands:

cd C:\Python27

.\env_tornado\Scripts\activate.bat

.\Scripts\easy_install.exe pip

pip.exe install tornado -E .\env_tornado

 

If you want to install in the global site packages, then skip pip above and use:

.\Scripts\easy_install.exe tornado

That’s it. Time to start writing code.

Setup virtualenv on Windows

I’m going to assume you have Python 2.7 correctly installed on Windows at C:\Python27. These instructions are applicable to Windows XP and Windows 7.

 

Open Windows PowerShell, then enter:

cd C:\Python27

.\Scripts\virtualenv.exe –no-site-packages env_tornado

And now to activate the new environment:

.\env_tornado\Scripts\activate.bat

The new virtual environment is now setup and ready for use

Output XML using Pyramid and Mako

I recently had to connect a .NET desktop application to a web service that was built using Pyramid. My first thought was to use JSONP. However, JSONP is only useful for small amounts of data…and I had a lot of data to send/receive. Obviously, communication via an XmlHttpRequest with JSON was not an option due to limitations imposed by the Same Origin Policy (ref: http://en.wikipedia.org/wiki/Same_origin_policy).

In the end, we chose to use our existing Mako template infrastructure to render XML that the .NET clients could consume. If you’re a .NET developer, then you know that XML parsing in .NET is superb, which is why we went with XML rather than JSON over HTTP.

On the Pyramid side, there were really only 2 steps:

  1. Create Mako templates that would render XML
  2. Set the content type to text/xml

Render XML with Mako

This step was easy, we just created templates that use XML instead of XML.

<?xml version=”1.0″ encoding=”utf-8″?>
<companies>
% if companies:
% for row in companies:
<company>
<id>${row[‘id’]}</id>
<name>${row[‘name’]}</name>
</company>
% endfor
% else:
${result}
% endif
</companies>

Set the Response Content Type

This was also a very easy step. We just set the content_type immediately before the call to return. Importantly, the code below causes Pyramid to return the XML template with a content type of text/xml rather than the default text/html.

request.response.content_type = “text/xml”
return {‘companies’: companies}

Summary

Rendering XML with Pyramid and Mako was exceptionally simple. You can reuse the same view and url, and simply change the the template based on how the view was called.

One last note. You’ll notice that one negative side effect we have not corrected for is the use of \n\r in the rendered XML. This of course bloats network traffic, uses more memory on the client-side (if you’re using a DOM parser), and is all around crappy for all but the smallest of files. Next step will be to create some type of processor to eliminate the newline character.

Naming standards and consistency

I’m a big proponent of keeping a consistent naming standard, even when crossing languages.

True, capitalization and syntax may change, but the standard should remain. Importantly, I have found that naming should play to the strengths of the operating system.

File Names

For example, on every operating system you can sort files by name (i.e. in Windows Explorer on Windows, Finder on Mac, and so on for Linux, etc.). With this in mind, naming should be used for a few key benefits:

  1. Group similar files
  2. Sort files by age/date

Grouping similar files is an important part of naming. For example, say we have a Users feature. We have a basic add, edit, update, delete and list all capability. In this situation, we want to preface our file names with user so that they are all sorted as a group. A good naming standard would be:

File Name Use
users List all Users
user_add Add a User
user_edit Edit a User
user_delete Delete a User
user_update Update a User

Notice how I did not specify a file name as this standard works well for HTML, CSS, JavaScript and Python files. To support other languages, we can simply tweak our standards to support the other language’s syntax, such as Users.vb, UserAdd.vb, UserEdit.vb and so on for Visual Basic.NET.

The above standard is superior to one where we put the action first, such as edit_user or add_user. If we put the action first, then all ‘edit’ files will be grouped together: edit_user, edit_company, edit_project.

The grouped approach is superior as we usually are working on the Users feature, or the Companies module, etc. By using the grouped approach all of our files are naturally and automatically sorted by the operating system.

HTML, CSS, JavaScript and Python

Similar to file names, functions, classes, styles, etc. should be consistently named throughout an application. For example, if we are creating a basic JSON enabled RESTful Web Service with JQuery and a JS library up front, then it’s a huge help to keep names consistent throughout the application (at all layers).

Building on our file names example above, the python def for adding a User should be:

def user_add(request):

Then in JavaScript our class definition would be for User with a User.add() method to save data to the server via basic CRUD Create operation.

User.add({

    name: ‘Bob’,

    age: ’23’,

})

Next, in HTML, we would have a form with a CSS class name that also fits our pattern:

<form class="user_add">

Summary

Consistent naming allows us to leverage built in operating systems for sorting files and reduces developer overhead. It makes it easier for software engineers at all layers to know what is above and below them in the software stack.