Production Deployment Using Apache, FastCGI and mod_rewrite, an alternate version

Amended to provide walkthoughs for two distinct configurations:

1. Apache.2.0.x: Apache version = 2.0.59, Python = 2.4, mod_fastcgi = mod_fastcgi-SNAP-0404142202
2. Apache.2.2.x: Apache version = 2.2.4, Python = 2.5, mod_fcgid = 2.1

This should be considered a work in progress, feel free to add/extend as necessary.

Organisation and bindings

Throughout the following, the Pylons app is called myapp, replace with your Pylons project name as appropriate.

This is based on existing, running installations of Apache+mods+Pylons: the Apache.2.0.x config on my CentOS 4.4-based technology lab server Bel-EPA and the Apache.2.2.x config on an in-house development Ubuntu-based system. I've tried to screen out as much of my own guff as I can whilst still leaving some context for reference.

The assumption is that Pylons apps live in /var/www/pylons/ and that there is legacy dynamic non-Pylons content that is to be served directly via Apache. It is assumed that the legacy content lives in /var/www/htdocs.

In this scheme, requests are routed by default to Pylons via a catch-all RewriteRule. Requests for legacy content are enabled via explicit RewriteCond statements that act as a guard for legacy content requests, preventing them from reaching the catch-all and allowing Apache to serve them "normally".

Procedure

Create a cache directory

Apache.2.0.x

Create and set permissions of cache directory in /tmp so that the webserver has read/write permissions.

1
2

% bash> sudo mkdir /tmp/fcgi_ipc
% bash> sudo chown webserver:webserver /tmp/fcgi_ipc

Apache.2.2.x

Create and set permissions of cache directories in /tmp so that the webserver has read/write permissions. Change"webserver:webserver" to match your server's user and group.

1
2
3

% bash> sudo mkdir /tmp/fcgid_sock/
% bash> sudo touch /tmp/fcgid_shm
% bash> sudo chown webserver:webserver /tmp/fcgid*

Apache preliminaries

Apache.2.0.x

First, you'll need to ensure you have both mod_rewrite and mod_fastcgi enabled. mod_rewrite is part of the standard Apache distro and may be readily available, if not actually enabled (have a look at the LoadModule statements in your Apache's httpd.conf file).

However, mod_fastcgi is not part of the standard Apache distro and will probably need to be downloaded from the fastcgi website, compiled and installed into /etc/httpd/modules. Note: I used the mod_fastcgi-SNAP-0404142202 download.

Apache.2.2.x

You'll need to have mod_fscgid enabled, so you will need to download, compile and install the mod_fcgid module.

If you can meet these starting conditions, you're ready to go ...

Adjust Apache configuration

Apache.2.0.x

Add the following to the LoadModule statements in /etc/httpd/conf/httpd.conf (assuming they're not there already):

1
2

LoadModule fastcgi_module modules/mod_fastcgi.so
LoadModule rewrite_module modules/mod_rewrite.so

I'm assuming a standard Apache setup here. If your module load path is different, just change the path as appropriate.

And add the following Directive to the others in the same file (/etc/httpd/conf/httpd.conf):

1
2
3
4
5
6
7

# [ Other directives ...]

<IfModule mod_fastcgi.c>
    FastCgiIpcDir /tmp/fcgi_ipc/
    AddHandler fastcgi-script .fcgi
</IfModule>
FastCgiExternalServer /var/www/pylons/myapp/myapp.fcgi -socket /var/www/pylons/myapp/myapp.socket

Set the following as the content of /etc/httpd/vhosts/myvhost.conf (changing or deleting the RewriteCond conditions as appropriate to your instance).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

<VirtualHost MY.IPA.DDR.ESS:80>
     ServerName myvhost.com
     ServerAdmin admin@myvhost.com
     DocumentRoot /var/www/htdocs

     # [ More directives ]

     RewriteEngine on
     RewriteCond %{REQUEST_URI} !^favicon.ico$
     RewriteCond %{REQUEST_URI} !^/robots.txt$
     RewriteCond %{REQUEST_URI} !^/error/.*
     RewriteCond %{REQUEST_URI} !^/icons/.*
     RewriteCond %{REQUEST_URI} !^/codebase/.*
     RewriteCond %{REQUEST_URI} !^/systems.html$
     RewriteCond %{REQUEST_URI} !^/systems/.*
     RewriteCond %{REQUEST_URI} !^/graphics/.*
     RewriteRule ^(.*)$ /myapp.fcgi$1 [L]

     <Location />
     	Options +ExecCGI
     	AddHandler fastcgi-script .fcgi
        AllowOverride All
     </Location>

     DirectoryIndex myapp.fcgi

     # Typical non-Pylons dynamic legacy content

     ScriptAlias /mailman/ /usr/lib/mailman/cgi-bin/
     <Directory /usr/lib/mailman/cgi-bin/>
         AllowOverride None
         Options ExecCGI
         Order allow,deny
         Allow from all
         AddHandler python-cgi-script .py
     </Directory>

     # *NB* This next Directive is specific to my server and 
     # my use of mod_pyapache, it's included here merely as 
     # an example and to provide some context.

     <Location /systems>
        <IfModule mod_pyapache.c>
           AddType application/x-httpd-cgi .py
           AddHandler application/x-python-httpd-cgi .py
           PythonPath /var/www/myapp/scripts
        </IfModule>
     </Location>

</VirtualHost>

<Directory /var/www/htdocs>
      order allow,deny
      allow from all
      Options Indexes All FollowSymLinks ExecCGI
      AllowOverride All
</Directory>

As an example, I've left in an IfModule directive (commented with "*NB*") that on my system enables mod_pyapache to execute Python source files dynamically but you won't need it (mod_pyapache no longer has effective support, it's legacy content, as mentioned earlier).

Apache.2.2.x

Add the following to the end of the list of LoadModule instructions in /etc/httpd/httpd.conf. (You can explicitly set the socket path and the shm area, otherwise the default is for them to be created automatically in the logs directory.)

1
2
3
4
5
6

LoadModule fcgid_module modules/mod_fcgid.so
<IfModule mod_fcgid.c>
    SocketPath /tmp/fcgid_sock/
    AddHandler fcgid-script .fcgi
    SharememPath /tmp/fcgid_shm
</IfModule>

Insert the following into the content of extra/httpd-vhost.conf (changing or deleting the RewriteCond conditions as appropriate to your instance).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

<VirtualHost *:80>
    ServerAdmin webmaster@myvhost.org
    DocumentRoot /var/www/pylons/myapp
    ServerName www.myvhost.org

    # Other directives
    # [ ... ]

    RewriteEngine On
    # RewriteLog logs/rewrite.log
    # A few other Apache-handled paths
    RewriteCond %{REQUEST_URI} !^/test/.*
    RewriteCond %{REQUEST_URI} !^/error/.*
    RewriteRule ^(.*)$ /myapp.fcgi$1 [L] 

    <Location />
        Options Indexes FollowSymLinks ExecCGI Includes
        Order allow,deny
        Allow from all
        AddHandler fcgid-script .fcgi
        FCGIWrapper "/usr/bin/python2.5 /var/www/pylons/myapp/myapp.fcgi" .fcgi
    </Location>

</VirtualHost>

<Directory /var/www/pylons>
    order allow,deny
    allow from all
    Options Indexes All FollowSymLinks ExecCGI Includes
    AllowOverride All
    <Files ~ "^\.ht">
        Order allow,deny
        Deny from all
    </Files>
</Directory>

Create a WSGI fcgi server

Create /var/www/pylons/myapp/myapp.fcgi with the following content:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

#!/usr/bin/env python
import logging
# Load the WSGI application from the config file
from paste.deploy import loadapp
wsgi_app = loadapp('config:/var/www/pylons/myapp/production.ini')

# Deploy it using FastCGI
if __name__ == '__main__':
     from flup.server.fcgi import WSGIServer
     WSGIServer(wsgi_app).run()

Mmmmm. Feel that power. Nearly done ...

Change your Pylons app's config to use PasteScript's flup_fcgi_thread

1. Make the following change to /var/www/pylons/myapp/production.ini — I've expressed the changes in diff format for clarity:

Apache.2.0.x

1
2
3
4
5
6

[server:main]
- use = egg:Paste#http
- host = 127.0.0.1
- port = 5000
+ use = egg:PasteScript#flup_fcgi_thread
+ socket = /var/www/pylons/myapp/myapp.socket

Apache.2.2.x

1
2
3
4
5

[server:main]
- use = egg:Paste#http
- host = 127.0.0.1
- port = 5000
+ use = egg:PasteScript#flup_fcgi_thread

That's about it.

Tweaking

For Apache.2.0.x,I ended up having to make a softlink from the fcgi script to the document root ...

1

% bash> sudo ln -s /var/www/pylons/myapp/myapp.fcgi /var/www/htdocs/

But this latter may be pure superstition, it took me a lot of faffing about to get this setup working and as soon as it was working, I sighed with relief and left it alone thereafter.

If you need to start/restart your Pylons app, you must do that by starting / restarting the Apache webserver.

Round-up

Where I am not trammelled by legacy content, I prefer to use mod_rewrite and ProxyPass / ProxyPassReverse to route all REQs to Pylons.

Feel free to add any comments or drop an email to gjh at bel-epa.com.